Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-32664

[2237678] VM snapshot is failing during fsfreeze with permission denied error

XMLWordPrintable

    • Storage Core Sprint 248, Storage Core Sprint 249, Storage Core Sprint 250, Storage Core Sprint 251, CNV I/U Operators Sprint 261
    • Medium
    • No

      Description of problem:

      If we mount a disk, by default it will have `unlabeled_t` SELinux context.

      ~~~

      1. mount /dev/sda /moun_point/
      1. ls -lZd /moun_point/
        drwxr-xr-x. 2 root root system_u:object_r:unlabeled_t:s0 6 Sep 6 06:12 /moun_point/
        ~~~

      Taking a snapshot of these VMs will fail during fsfreeze phase:

      ~~~

      {"component":"virt-handler","kind":"","level":"error","msg":"Failed to freeze VMI","name":"rhel8-5px23yqqlrnm0s1w","namespace":"new-nijin-cnv","pos":"lifecycle.go:124","reason":"server error. command Freeze failed: \"LibvirtError(Code=1, Domain=10, Message='internal error: unable to execute QEMU agent command 'guest-fsfreeze-freeze': failed to open /moun_point: Permission denied')\"","timestamp":"2023-09-06T10:16:22.703074Z","uid":"b5bbd812-18a7-495a-93fa-9172dd9375f5"}

      ~~~

      The qemu-ga won't be able to work on unlabeled_t directories:

      ~~~
      Sep 06 06:16:33 rhel8-5px23yqqlrnm0s1w setroubleshoot[5401]: SELinux is preventing /usr/bin/qemu-ga from read access on the directory /moun_point. For >

      Then you can run restorecon. The access attempt may have been stopped due to insufficient >
      Do

      1. /sbin/restorecon -v /moun_point

      If you want to allow virt to qemu ga read nonsecurity files
      Then you must tell SELinux about this by enabling the 'virt_qemu_ga_read_nonsecurity_files>

      Do
      setsebool -P virt_qemu_ga_read_nonsecurity_files 1
      ~~~

      As mentioned in the message, we need to either fix the context or set boolean virt_qemu_ga_read_nonsecurity_files to 1 to fix the problem.

      Version-Release number of selected component (if applicable):

      OpenShift Virtualization 4.13.3

      How reproducible:

      100%

      Steps to Reproduce:

      1. Create a RHEL 8 or 9 VM from a template.
      2. Add a new disk, format it, and mount it on a directory.
      3. Try to take a snapshot of the VM.
      4. It will be in "progress" state for some time and will finally fail.

      Actual results:

      VM snapshot is failing during fsfreeze with permission denied error.

      Expected results:

      I am not sure where it should be fixed. The boolean virt_qemu_ga_read_nonsecurity_files was implemented in bug 1747960 and it is false by default. In the case of RHV, the snapshot does not fail even if fsfreeze fails, and the snapshot operation completes with a warning. In the case of OpenShift Virtualization, the user is clueless about the failure and has to look at the logs to understand the problem. I think it would be helpful if we could pick up the error and show the recommendation to the user to correct the context.

      Additional info:

              rh-ee-alromero Alvaro Romero
              rhn-support-nashok Nijin Ashok
              Dalia Frank Dalia Frank
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: