Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-54193

guest-fsfreeze-freeze call in Windows VM returns error: "Guest agent is not responding"

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • CNV v4.21.0
    • CNV v4.16.0, CNV v4.16.5
    • Storage Ecosystem
    • None
    • Incidents & Support
    • 1
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • CNV Storage 269, CNV Storage 270
    • Important
    • None

      Description of problem:

      We have seen that OADP backups of Windows 2022 and 2016 VMs fail from time to time with an error when freezing the fs:
      
      time="2024-12-23T12:51:08Z" level=info msg="stderr: {\"component\":\"freezer\",\"level\":\"info\",\"msg\":\"Starting...\",\"pos\":\"main.go:59\",\"timestamp\":\"2024-12-23T12:51:03.121845Z\"}\n{\"component\":\"freezer\",\"level\":\"info\",\"msg\":\"Guest agent version is 108.0.1\",\"pos\":\"main.go:102\",\"timestamp\":\"2024-12-23T12:51:03.125041Z\"}\n{\"component\":\"freezer\",\"level\":\"error\",\"msg\":\"Freezeing VMI failed\",\"pos\":\"main.go:112\",\"reason\":\"server error. command Freeze failed: \\\"LibvirtError(Code=86, Domain=10, Message='Guest agent is not responding: Guest agent not available for now')\\\"\",\"timestamp\":\"2024-12-23T12:51:08.145052Z\"}\n" backup=openshift-adp/backup-case-03980918-04 hookCommand="[/usr/bin/virt-freezer --freeze --name windows-2k22-virtio-crimson-worm-76 --namespace default]" hookContainer=compute hookName="<from-annotation>" hookOnError=Fail hookPhase=pre hookSource=annotation hookTimeout="{30s}" hookType=exec logSource="/remote-source/velero/app/pkg/podexec/pod_command_executor.go:181" name=virt-launcher-windows-2k22-virtio-crimson-worm-76-bmxnl namespace=default resource=pods
      time="2024-12-23T12:51:08Z" level=error msg="Error executing hook" backup=openshift-adp/backup-case-03980918-04 error="command terminated with exit code 1" hookPhase=pre hookSource=annotation hookType=exec logSource="/remote-source/velero/app/internal/hook/item_hook_handler.go:239" name=virt-launcher-windows-2k22-virtio-crimson-worm-76-bmxnl namespace=default resource=pods
      
      
      The problem is that the guest agent call 'guest-fsfreeze-freeze' returns error:
      
      $ oc exec virt-launcher-windows-2k22-virtio-crimson-worm-76-z4mhk -- virsh qemu-agent-command 1 '{"execute": "guest-fsfreeze-freeze"}'
      
      error: Guest agent is not responding: Guest agent not available for now
      command terminated with exit code 1
      

      Version-Release number of selected component (if applicable):

      kubevirt-hyperconverged-operator.v4.16.5
      QEMU Guest Agent 108.0.1

      How reproducible:

      Most of the time, but not always.

      Steps to Reproduce:

      1. Install Windows Server 2022 with virtio devices (windows.2k22.virtio Preference)
      2. Install virtio drivers and qemu-ga from the ISO provided by OCP
      3. Run fsfreeze in the guest. Can also be triggered by creating a VM snapshot:
      
      $ oc exec virt-launcher-windows-2k22-virtio-crimson-worm-76-z4mhk -- virsh qemu-agent-command 1 '{"execute": "guest-fsfreeze-freeze"}'
      error: Guest agent is not responding: Guest agent not available for now
      command terminated with exit code 1
      
      $ oc exec virt-launcher-windows-2k22-virtio-crimson-worm-76-z4mhk -- virsh qemu-agent-command 1 '{"execute": "guest-fsfreeze-status"}'
      {"return":"frozen"}
      
      $ oc exec virt-launcher-windows-2k22-virtio-crimson-worm-76-z4mhk -- virsh qemu-agent-command 1 '{"execute": "guest-fsfreeze-thaw"}'
      {"return":3}
      

      Actual results:

      'guest-fsfreeze-freeze' call returns error 1
      
      Apparently the filesystems are frozen correctly and no errors can be seen in the Windows event log.

      Expected results:

      No errors in the freeze call.

      Additional info:

       

              skagan@redhat.com Shelly Kagan
              rhn-support-jortialc Juan Orti
              Satheesaran Sundaramoorthi Satheesaran Sundaramoorthi
              Votes:
              2 Vote for this issue
              Watchers:
              9 Start watching this issue

                Created:
                Updated: