Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-3373

Ceilometer compute agent is unable to connect to libvirt socket

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • ?
    • ?
    • openstack-selinux-0.8.40-18.0.20240418104643.f618d90.el9ost
    • ?
    • ?
    • No
    • Release Note Not Required
    • Important

      I discovered, that ceilometer compute agent container is no longer able to connect to the /var/run/libvirt/libvirt-sock-ro socket. This is possibly due to a recent change, after which libvirt is installed locally on the compute node instead of being in a container.

      Logs from ceilometer compute agent container after container starts (this repeats a bunch of times as it tries to connect to the socket when loading each extension)

      2024-01-12 13:59:11.013 9 DEBUG ceilometer.compute.virt.libvirt.utils [-] Connecting to libvirt: qemu:///system new_libvirt_connection /usr/lib/python3.9/site-packages/ceilometer/compute/virt/libvirt/utils.py:93
      libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/virtqemud-sock-ro': Permission denied
      2024-01-12 13:59:11.013 9 DEBUG ceilometer.polling.manager [-] Skip loading extension for network.outgoing.packets.error: Failed to connect socket to '/var/run/libvirt/virtqemud-sock-ro': Permission denied _catch_extension_load_error /usr/lib/python3.9/site-packages/ceilometer/polling/manager.py:421
       

      Part of /var/log/audit/audit.log after restarting ceilometer container with SELinux in permissive mode:

      type=SERVICE_START msg=audit(1705066462.342:25823): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=edpm_ceilometer_agent_compute comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset"
      type=AVC msg=audit(1705066464.562:25824): avc:  denied  { associate } for  pid=101127 comm="ceilometer-poll" name="1" scontext=system_u:object_r:container_t:s0:c299,c367 tcontext=system_u:object_r:proc_t:s0 tclass=filesystem permissive=1
      type=SYSCALL msg=audit(1705066464.562:25824): arch=c000003e syscall=257 success=yes exit=8 a0=ffffff9c a1=7f6172514530 a2=80441 a3=1b6 items=1 ppid=101125 pid=101127 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ceilometer-poll" exe="/usr/bin/python3.9" subj=system_u:system_r:container_t:s0:c299,c367 key=(null)ARCH=x86_64 SYSCALL=openat AUID="unset" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"
      type=CWD msg=audit(1705066464.562:25824): cwd="/"
      type=PATH msg=audit(1705066464.562:25824): item=0 name="/dev/stdout" inode=641820 dev=00:0d mode=010777 ouid=0 ogid=0 rdev=00:00 obj=system_u:system_r:container_runtime_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0OUID="root" OGID="root"
      type=PROCTITLE msg=audit(1705066464.562:25824): proctitle=6365696C6F6D657465722D706F6C6C696E673A206D61737465722070726F63657373205B2F7573722F62696E2F6365696C6F6D657465722D706F6C6C696E67202D2D706F6C6C696E672D6E616D6573706163657320636F6D70757465202D2D6C6F6766696C65202F6465762F7374646F75745D
      type=AVC msg=audit(1705066464.668:25825): avc:  denied  { connectto } for  pid=101135 comm="ceilometer-poll" path="/run/libvirt/virtqemud-sock-ro" scontext=system_u:system_r:container_t:s0:c299,c367 tcontext=system_u:system_r:virtd_t:s0-s0:c0.c1023 tclass=unix_stream_socket permissive=1
      type=SYSCALL msg=audit(1705066464.668:25825): arch=c000003e syscall=42 success=yes exit=0 a0=7 a1=7f617249c360 a2=6e a3=1 items=0 ppid=101127 pid=101135 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ceilometer-poll" exe="/usr/bin/python3.9" subj=system_u:system_r:container_t:s0:c299,c367 key=(null)ARCH=x86_64 SYSCALL=connect AUID="unset" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"
      type=PROCTITLE msg=audit(1705066464.668:25825): proctitle=6365696C6F6D657465722D706F6C6C696E673A206D61737465722070726F63657373205B2F7573722F62696E2F6365696C6F6D657465722D706F6C6C696E67202D2D706F6C6C696E672D6E616D6573706163657320636F6D70757465202D2D6C6F6766696C65202F6465762F7374646F75745D
      type=SERVICE_START msg=audit(1705066465.349:25826): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=setroubleshootd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'UID="root" AUID="unset" 

      Output of audit2allow -a

      #============= container_t ==============
      allow container_t proc_t:filesystem associate;
      allow container_t virtd_t:unix_stream_socket connectto; 

      NOTE: We are already starting the ceilometer container with `–volume /run/libvirt:/run/libvirt:shared,z` This worked the last time I checked if the container works a few months ago.

        1. audit.log
          1.53 MB
          Juan Larriba
        2. audit.tar.gz
          435 kB
          Juan Larriba

              rh-ee-jwysogla Jaromir Wysoglad
              rh-ee-jwysogla Jaromir Wysoglad
              rhos-dfg-reldel
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

                Created:
                Updated:
                Resolved: