Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-39893

AAQ: PODs remain in gated state after removing label from the namespace

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Major Major
    • CNV v4.16.0
    • None
    • CNV Virtualization
    • None
    • 2
    • False
    • Hide

      None

      Show
      None
    • False
    • aaq-operator-rhel9-container-v4.16.0-77
    • ---
    • ---
    • 2
    • CNV Virtualization Sprint 251
    • No

      Description of problem:

      After removing the label from the namespace or removing ACRQ from cluster PODs continue to stay in the SchedulingGated state until at least one of the VMs is restarted:
      
      # ACRQ with label selector and quota pods=1
        total:
          hard:
            pods: "1"
          used:
            pods: "1"
      
      # One VM is running, other PODs in gated state
      $ oc get pod
      NAME                       READY   STATUS            RESTARTS   AGE
      virt-launcher-vm-a-gssc2   0/2     SchedulingGated   0          6m31s
      virt-launcher-vm-b-j9vgd   2/2     Running           0          6m31s
      virt-launcher-vm-c-nszx8   0/2     SchedulingGated   0          6m31s
      virt-launcher-vm-d-htb2h   0/2     SchedulingGated   0          6m30s
      virt-launcher-vm-e-cvpq8   0/2     SchedulingGated   0          6m30s
      
      # Removed label from namespace - PODs are still in gated state:
      $ oc get pod
      NAME                       READY   STATUS            RESTARTS   AGE
      virt-launcher-vm-a-gssc2   0/2     SchedulingGated   0          17m
      virt-launcher-vm-b-j9vgd   2/2     Running           0          17m
      virt-launcher-vm-c-nszx8   0/2     SchedulingGated   0          17m
      virt-launcher-vm-d-htb2h   0/2     SchedulingGated   0          17m
      virt-launcher-vm-e-cvpq8   0/2     SchedulingGated   0          17m
      
      # restarted one of VMs and all of them were ungated
      $ virtctl restart vm-a
      VM vm-a was scheduled to restart
      
      $ oc get pod
      NAME                       READY   STATUS            RESTARTS   AGE
      virt-launcher-vm-a-fk4w9   0/2     Init:0/2          0          3s
      virt-launcher-vm-b-j9vgd   2/2     Running           0          18m
      virt-launcher-vm-c-nszx8   2/2     PodInitializing   0          18m
      virt-launcher-vm-d-htb2h   0/2     PodInitializing   0          18m
      virt-launcher-vm-e-cvpq8   0/2     PodInitializing   0          18m

      Version-Release number of selected component (if applicable):

      4.15

      How reproducible:

      100%

      Steps to Reproduce:

      1. create acrq with label selector and pods quota
      2. create namespace with this label
      3. create multiple VMs and check virt-launcher pods are in SchedulingGated state
      4. remove label from namespace (or delete ACRQ)
      
      

      Actual results:

      PODs remain in SchedulingGated state

      Expected results:

      PODs should run as soon as quota removed

      Additional info:

       

            bmordeha@redhat.com Barak Mordehai
            dshchedr@redhat.com Denys Shchedrivyi
            Kedar Bidarkar Kedar Bidarkar
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: