Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-34712

[2247157] CNV DataSources taking >90 minutes to import and VM PVCs are taking >30 minutes to clone

XMLWordPrintable

    • Storage Core Sprint 248
    • Medium
    • No

      Created attachment 1996303 [details]
      cdi-deployment pod log

      Description of problem:
      Upon vanilla installation of OpenShift 4.13.17 SNO with OCP Virt and LVM Storage operators installed as part of Assisted Installer cluster creation, the data sources take up to 90 minutes to finish importing with upwards of 15 restarts over that period. The topolvm-node-* will enter a crashloopbackoff. Manually deleting the pod and allowing for recreation will rectify the crashloopbackoff situation. Finally, the cloning of a PVC for a newly created VM takes upwards of 30 minutes.

      Version-Release number of selected component (if applicable):
      OpenShift Virtualization - 4.13.4
      LVM Storage - 4.14.0
      OpenShift - 4.13.17

      How reproducible:
      Every time on test cluster

      Steps to Reproduce:
      1. Deploy cluster on bare metal server (48vCPUs, 256GB RAM, 2TB SSD storage on 2 1TB SSDs) using interactive installer, selecting SNO, static IP configuration, and OCPvirt and LVM Storage selected
      2. Cluster deploys in about 30 minutes

      Actual results:
      [chad@bastion ~]$ oc get -n openshift-virtualization-os-images dv
      NAME PHASE PROGRESS RESTARTS AGE
      centos-stream9-aea06c312f87 ImportInProgress 99.91% 19 105m
      rhel8-2cde3f47f8c7 ImportInProgress 0.00% 19 105m
      rhel9-a1947a1edca5 ImportInProgress 0.00% 19 105m

      [chad@bastion ~]$ oc get -n openshift-virtualization-os-images pods
      NAME READY STATUS RESTARTS AGE
      importer-centos-stream9-aea06c312f87 1/2 CrashLoopBackOff 21 (17s ago) 110m
      importer-rhel8-2cde3f47f8c7 2/2 Running 20 (6m23s ago) 111m
      importer-rhel9-a1947a1edca5 1/2 CrashLoopBackOff 20 (65s ago) 111m

      Eventually, all will finish their import. I then create a VM, in this case Centos9
      [chad@bastion ~]$ oc get pods -n vm-test
      NAME READY STATUS RESTARTS AGE
      cdi-upload-centos-stream9-empty-skunk 0/1 Running 0 10s

      [chad@bastion ~]$ oc get dv -n vm-test
      NAME PHASE PROGRESS RESTARTS AGE
      centos-stream9-empty-skunk CloneInProgress 4.71% 2 2m58s

      This took approximately 30 minutes, I do not know how to search for that particular metric.

      Expected results:
      A VM deploys on OCP SNO without error and without import and cloning errors that take over 2 hours to overcome.

      Additional info:
      I have additional logs and can provide any resources necessary regarding the test cluster.

              akalenyu Alex Kalenyuk
              chobbs-sa Chad Hobbs
              Natalie Gavrielov Natalie Gavrielov
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: