Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-32937

[2238786] After draining node where mtq system pods running the namespace becomes locked but ResourceQuota not updated

XMLWordPrintable

    • High
    • No

      Description of problem:
      When I drained the node where mtq-lock and mtq-controller pod were running - namespace becomes locked, but ResourceQuota was not increased. As result - migration stuck in Pending state:

      > $ oc get vmim
      > NAME PHASE VMI
      > kubevirt-evacuation-s5sft Pending vm-fedora-1

      > $ oc get resourcequota
      > NAME AGE REQUEST LIMIT
      > quota-cpu 16h requests.cpu: 1001m/1100m limits.cpu: 1010m/1100m

      ValidatingWebhookConfiguration is present (the namespace is locked):
      > $ oc get ValidatingWebhookConfiguration | grep lock
      > lock.default.com 1 4m58s

      and I can't delete vmmrq:
      > $ oc delete vmmrq my-vmmrq
      > Error from server: admission webhook "lock.default.com" denied the request: Migration process is currently being handled by the Managed Quota controller, and as a result, modifications,creation or deletion of virtualMachineMigrationResourceQuotas are not permitted in this namespace, please try again.

      after manually removing lock and recreating vmmrq - migration started

      Version-Release number of selected component (if applicable):
      4.14

      How reproducible:

      Steps to Reproduce:
      1. get mtq-controller and mtq-lock pods running on one node
      2. create and run VM on the same node
      3. drain that node, for example:
      oc adm drain --delete-local-data --ignore-daemonsets=true --force virt-den-415-8gq52-worker-0-9wz8h

      Actual results:
      Migration does not work

      Expected results:
      VM successfully migrated during the node drain

      Additional info:

            bmordeha@redhat.com Barak Mordehai
            dshchedr@redhat.com Denys Shchedrivyi
            Kedar Bidarkar Kedar Bidarkar
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: