Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-39141

Downloading virtctl memory-dump results in empty tar file.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Major Major
    • CNV v4.17.0
    • None
    • Storage Platform
    • None
    • 5
    • False
    • Hide

      None

      Show
      None
    • False
    • CNV v4.17.0.rhel9-60
    • ---
    • ---
    • Storage Core Sprint 252, Storage Core Sprint 253, Storage Core Sprint 254, Storage Core Sprint 255, Storage Core Sprint 258
    • High
    • No

      Description of problem:

      Attempting to download a Memory Dump from a VM results in an empty tar directory. 

      Version-Release number of selected component (if applicable):

      4.12 and 4.13

      How reproducible:

      Steps to Reproduce:

      On 4.12:
      
      Create dump with: 
      #virtctl memory-dump get rhel8-feqarbepxe4sgxcf --claim-name=shaggy-dump --create-claim
      
      
      $oc describe vm rhel8-feqarbepxe4sgxcf 
         Memory Dump Request:
          Claim Name:       shaggy-dump
          End Timestamp:    2024-03-07T18:56:21Z
          File Name:        rhel8-feqarbepxe4sgxcf-shaggy-dump-20240307-185618.memory.dump
          Phase:            Completed
          Start Timestamp:  2024-03-07T18:56:18Z
      
      $oc get pvc
      NAME                     STATUS   VOLUME                                     CAPACITY      ACCESS MODES   STORAGECLASS   AGE
      shaggy-dump              Bound    pvc-5c2b9e8e-cab3-42f0-b00d-06332072e5cb   2267Mi        RWO            nfs-client     35s
      
      
      Download it with: 
      #virtctl vmexport download shaggy-dump --pvc=shaggy-dump --volume=pvc-5c2b9e8e-cab3-42f0-b00d-06332072e5cb --output=shaggy.img.gz
      
      However it looks to be empty? 
      [kni@cee ~]$ tar -ztvf shaggy.img.gz
      drwxrwxrwx root/root         0 2024-03-07 13:56 ./
      
      
      

       

      On 4.13 using the all in one command:
      
      $ virtctl memory-dump get rhel8-dumptest --create-claim --claim-name=dumptest --output=dumptest.tar.gz
      PVC shaggy/dumptest created
      Successfully submitted memory dump request of VM rhel8-dumptest
      Waiting for memorydump  to complete, current phase: Associating...
      Waiting for memorydump  to complete, current phase: InProgress...
      Waiting for memorydump  to complete, current phase: InProgress...
      Memory dump completed successfully
      VirtualMachineExport 'shaggy/export-rhel8-dumptest-dumptest' created succesfully
      Downloading file: 122 B [___________________] ? p/s                                                                                                                                                
      Download finished succesfully
      VirtualMachineExport 'shaggy/export-rhel8-dumptest-dumptest' deleted succesfully
      
      $ tar -ztvf dumptest.tar.gz 
      drwxrwxrwx virt-exportserver/107 0 2024-03-07 14:15 ./
      
      Above downloaded dump is empty also
      
      
      $oc describe vm rhel8-dumptest
      
        Memory Dump Request:
          Claim Name:       dumptest
          End Timestamp:    2024-03-07T19:15:36Z
          File Name:        rhel8-dumptest-dumptest-20240307-191534.memory.dump
          Phase:            Completed
          Start Timestamp:  2024-03-07T19:15:34Z
      
      
      Attach dumptest pvc to vm in the GUI, stop VM, start VM 
      
      [kni@cee ~]$ oc get pods -o wide
      NAME                                 READY   STATUS    RESTARTS   AGE   IP            NODE                 NOMINATED NODE   READINESS GATES
      hp-volume-9dkqd                      1/1     Running   0          7s    10.128.3.25   openshift-worker-1   <none>           <none>
      virt-launcher-rhel8-dumptest-rs5pp   1/1     Running   0          10s   10.128.3.24   openshift-worker-1   <none>           1/1
      
      
      oc debug node/openshift-worker-1 
      we see the dump pvc mounted here on the host: 
      
      sh-4.4# df | grep dumptest
      192.168.111.1:/shaggy-rhel8-dumptest-pvc-0a340671-13fd-4e2e-b2ab-8b45d3bbf8a2 3669661696 326250496 3343411200   9% /host/var/lib/kubelet/pods/0d24eb5d-9f66-4262-99c1-d75b098fa144/volumes/kubernetes.io~nfs/pvc-0a340671-13fd-4e2e-b2ab-8b45d3bbf8a2
      192.168.111.1:/shaggy-dumptest-pvc-d5a7ac30-5c00-4ab4-870c-b72effdce7e7       3669661696 326250496 3343411200   9% /host/var/lib/kubelet/pods/4434f45e-b5e7-47af-a148-a23ef5520edf/volumes/kubernetes.io~nfs/pvc-d5a7ac30-5c00-4ab4-870c-b72effdce7e7    
      
      
      sh-4.4# cd /host/var/lib/kubelet/pods/4434f45e-b5e7-47af-a148-a23ef5520edf/volumes/kubernetes.io~nfs/pvc-d5a7ac30-5c00-4ab4-870c-b72effdce7e7
      sh-4.4# ls -la
      total 2113732
      drwxrwxrwx. 2  107  107         65 Mar  7 19:15 .
      drwxr-x---. 3 root root         54 Mar  7 19:17 ..
      -rw-------. 1  107  107 2164458827 Mar  7 19:15 rhel8-dumptest-dumptest-20240307-191534.memory.dump
      
      Here we see the file that we would need for the dump, so we know the dump was created successfully
      
      

       

      Actual results:

      Downloaded tar file is empty and only 122B

      Expected results:

      The tar file should contain the entire memory image and be approx 2GB to match the amount of memory 

              rh-ee-alromero Alvaro Romero
              shaselde@redhat.com Sean Haselden
              Kevin Alon Goldblatt Kevin Alon Goldblatt
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

                Created:
                Updated:
                Resolved: