Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-7316

stap --remote libvirt:// does not work as expected

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • rhel-9.0.0
    • systemtap
    • rhel-sst-pt-perf-debug
    • ssg_platform_tools
    • 3
    • False
    • Hide

      None

      Show
      None
    • No
    • None
    • None
    • None
    • If docs needed, set a value
    • None

      Description of problem:
      Execute from HOST
      Works well if we use ssh connection

      1. stap --remote-prefix --remote root@100.100.100.30 -ve 'probe begin { printf("Hello from VM3-RHEL9-DEV!\n") }'
        Pass 1: parsed user script and 496 library scripts using 387992virt/148384res/16740shr/131468data kb, in 430usr/50sys/484real ms.
        Pass 2: analyzed script: 1 probe, 0 functions, 0 embeds, 0 globals using 392216virt/153116res/17004shr/135692data kb, in 30usr/0sys/32real ms.
        Pass 3: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.c
        Pass 4: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.ko
        Pass 5: starting run.
        0: Hello from VM3-RHEL9-DEV! <<== WE CAN SEE HELLO
        ^CPass 5: run completed in 20usr/0sys/3359real ms.


        Can not work is we use libvirt
        # stap --remote-prefix --remote libvirt://VM3-RHEL9-DEV -ve 'probe begin { printf("Hello from VM3-RHEL9-DEV!n") }

        '
        Pass 1: parsed user script and 496 library scripts using 388028virt/148540res/16860shr/131504data kb, in 420usr/60sys/486real ms.
        Pass 2: analyzed script: 1 probe, 0 functions, 0 embeds, 0 globals using 392252virt/153212res/17060shr/135728data kb, in 40usr/10sys/33real ms.
        Pass 3: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.c
        Pass 4: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.ko
        Pass 5: starting run.
        <<== NOTHING HAPPEN WE CAN NOT SEE HELLO

      Version-Release number of selected component (if applicable):

      HOST INFO
      systemtap-runtime-virthost-4.6-12.el9_0.x86_64
      systemtap-sdt-devel-4.6-12.el9_0.x86_64
      systemtap-runtime-4.6-12.el9_0.x86_64
      systemtap-client-4.6-12.el9_0.x86_64
      systemtap-devel-4.6-12.el9_0.x86_64
      systemtap-4.6-12.el9_0.x86_64
      systemtap-server-4.6-12.el9_0.x86_64

      libvirt-libs-8.0.0-8.el9_0.x86_64
      libvirt-glib-4.0.0-3.el9.x86_64
      libvirt-dbus-1.4.1-5.el9.x86_64
      libvirt-client-8.0.0-8.el9_0.x86_64
      python3-libvirt-8.0.0-1.el9.x86_64
      libvirt-daemon-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-storage-core-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-network-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-nwfilter-8.0.0-8.el9_0.x86_64
      libvirt-daemon-config-nwfilter-8.0.0-8.el9_0.x86_64
      libvirt-daemon-config-network-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-storage-disk-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-storage-mpath-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-storage-scsi-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-storage-iscsi-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-storage-rbd-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-storage-logical-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-storage-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-interface-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-qemu-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-secret-8.0.0-8.el9_0.x86_64
      libvirt-daemon-driver-nodedev-8.0.0-8.el9_0.x86_64
      libvirt-8.0.0-8.el9_0.x86_64

      VM3-RHEL9-DEV INFO

      systemtap-runtime-4.6-12.el9_0.x86_64
      systemtap-client-4.6-12.el9_0.x86_64
      systemtap-devel-4.6-12.el9_0.x86_64
      systemtap-4.6-12.el9_0.x86_64
      systemtap-server-4.6-12.el9_0.x86_64
      systemtap-runtime-virtguest-4.6-12.el9_0.x86_64
      systemtap-initscript-4.6-12.el9_0.x86_64
      systemtap-runtime-virthost-4.6-12.el9_0.x86_64
      systemtap-debugsource-4.6-12.el9_0.x86_64
      systemtap-debuginfo-4.6-12.el9_0.x86_64
      systemtap-runtime-debuginfo-4.6-12.el9_0.x86_64

      How reproducible:
      Execute command
      Can not work is we use libvirt

      1. stap --remote-prefix --remote libvirt://VM3-RHEL9-DEV -ve 'probe begin { printf("Hello from VM3-RHEL9-DEV!\n") }'
        Pass 1: parsed user script and 496 library scripts using 388028virt/148540res/16860shr/131504data kb, in 420usr/60sys/486real ms.
        Pass 2: analyzed script: 1 probe, 0 functions, 0 embeds, 0 globals using 392252virt/153212res/17060shr/135728data kb, in 40usr/10sys/33real ms.
        Pass 3: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.c
        Pass 4: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.ko
        Pass 5: starting run.
        <<== NOTHING HAPPEN WE CAN NOT SEE HELLO

        Steps to Reproduce:
        1.
        2.
        3.

        Actual results:


        Expected results:
        It should work if use libvirt connection and we should see the output

        Additional info:
        Execute from HOST
        Works well if we use ssh connection
        # stap --remote-prefix --remote root@100.100.100.30 -ve 'probe begin { printf("Hello from VM3-RHEL9-DEV!n") }

        '
        Pass 1: parsed user script and 496 library scripts using 387992virt/148384res/16740shr/131468data kb, in 430usr/50sys/484real ms.
        Pass 2: analyzed script: 1 probe, 0 functions, 0 embeds, 0 globals using 392216virt/153116res/17004shr/135692data kb, in 30usr/0sys/32real ms.
        Pass 3: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.c
        Pass 4: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.ko
        Pass 5: starting run.
        0: Hello from VM3-RHEL9-DEV! <<== WE CAN SEE HELLO
        ^CPass 5: run completed in 20usr/0sys/3359real ms.

      Can not work is we use libvirt

      1. stap --remote-prefix --remote libvirt://VM3-RHEL9-DEV -ve 'probe begin { printf("Hello from VM3-RHEL9-DEV!\n") }

        '
        Pass 1: parsed user script and 496 library scripts using 388028virt/148540res/16860shr/131504data kb, in 420usr/60sys/486real ms.
        Pass 2: analyzed script: 1 probe, 0 functions, 0 embeds, 0 globals using 392252virt/153212res/17060shr/135728data kb, in 40usr/10sys/33real ms.
        Pass 3: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.c
        Pass 4: using cached /root/.systemtap/cache/f2/stap_f2f4984a8595cb3047ff21e278bfd589_877.ko
        Pass 5: starting run.
        <<== NOTHING HAPPEN WE CAN NOT SEE HELLO

      stapvirt info

      1. stapvirt query VM3-RHEL9-DEV

      Name: VM3-RHEL9-DEV
      UUID: a719c10c-0b9b-4e3d-bfa8-af5bee183e44
      State: running
      ID: 63
      Type: persistent
      Permanent Ports: 2
      Hotplugging: supported

      1. stapvirt port-list VM3-RHEL9-DEV
        /var/lib/libvirt/qemu/VM3-RHEL9-DEV.org.systemtap.stapsh.0.sock
        /var/lib/libvirt/qemu/VM3-RHEL9-DEV.org.systemtap.stapsh.1.sock

      On VM3-RHEL9-DEV

      1. systemctl status stapsh@org.systemtap.stapsh.0.service
        ● stapsh@org.systemtap.stapsh.0.service - SystemTap stapsh on org.systemtap.stapsh.0
        Loaded: loaded (/usr/lib/systemd/system/stapsh@.service; static)
        Active: active (running) since Tue 2022-06-14 14:41:10 CST; 52min ago
        Docs: man:stap
        man:stapsh
        Main PID: 73714 (stapsh)
        Tasks: 1 (limit: 20701)
        Memory: 176.0K
        CPU: 30ms
        CGroup: /system.slice/system-stapsh.slice/stapsh@org.systemtap.stapsh.0.service
        └─73714 /usr/bin/stapsh -l /dev/virtio-ports/org.systemtap.stapsh.0

      Jun 14 14:41:10 RHEL9DEV systemd[1]: Started SystemTap stapsh on org.systemtap.stapsh.0.

      #lspci |grep console
      09:00.0 Communication controller: Red Hat, Inc. Virtio console (rev 01)

              rhtorsys Frank Eigler
              rhn-support-xili Xinhua Li
              Frank Eigler Frank Eigler
              qe-baseos-tools-bugs@redhat.com qe-baseos-tools-bugs@redhat.com qe-baseos-tools-bugs@redhat.com qe-baseos-tools-bugs@redhat.com
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated: