Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-7977

Quay 3.12 Operator should not reconcile to update minReplicas for Clair-APP and Mirror when HPA is set to unmanaged

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • quay-v3.12.6
    • quay-v3.12.3
    • quay
    • False
    • None
    • False

      Description:

      This is an issue found in Quay 3.12.3, when deploy Quay with managed HPA, then update the QuayRegistry CR to use unmanaged HPA, Quay Operator will continue to terminate and reset the minReplicas to default value 2 for Mirror and Clair-App HPA, this issue is not existed with Quay APP HPA, pls review this issue.

      Quay: 3.12.3(quay-operator-bundle-container-v3.12.2-3)

      Steps:

      1. Deploy Quay with Quay Operator with managed HPA
      2. Edit the Quay CRD and set the HPA as "managed: false"
      3. Edit the "minReplicas" for Quay, Mirror and Clair HPA to set as customized value(e.g 5)
      4. Check Quay, Mirror and Clair deployments.
      5. The results is Quay Operator will not terminate Quay APP PODS
      6. The results is Quay Operator will terminate Mirror and Clair PODS, then the HPA will provisioning new PODS to meet the minReplicas, but Quay Operator will terminate PODS to set the minReplicas to 2 again and again.
      Quay 3.12.3 Operator continue to terminate the PODS of Mirror and Clair-App PODS:

      oc get pod
      NAME                      READY  STATUS    RESTARTS  AGE
      quay-operator.v3.12.3-7769ddb5d-9zq6b      1/1   Running    0     112m
      quay3123hcpcs-clair-app-7b75b8d464-tg6j2    1/1   Running    0     14m
      quay3123hcpcs-clair-app-7b75b8d464-whd7j    1/1   Running    0     14m
      quay3123hcpcs-clair-postgres-6544687f48-njmkq  1/1   Running    0     62m
      quay3123hcpcs-quay-app-85bfd96dbf-5qn8l     1/1   Running    0     14m
      quay3123hcpcs-quay-app-85bfd96dbf-622v8     1/1   Running    0     3m11s
      quay3123hcpcs-quay-app-85bfd96dbf-9frc4     1/1   Running    0     3m11s
      quay3123hcpcs-quay-app-85bfd96dbf-m2tjd     1/1   Running    0     13m
      quay3123hcpcs-quay-app-85bfd96dbf-zkt2p     1/1   Running    0     3m11s
      quay3123hcpcs-quay-app-upgrade-g9b6l      0/1   Completed   2     66m
      quay3123hcpcs-quay-database-cd9cfbd6d-rmvzx   1/1   Running    0     65m
      quay3123hcpcs-quay-mirror-5f68595776-flfn4   1/1   Running    0     14m
      quay3123hcpcs-quay-mirror-5f68595776-s9nnr   1/1   Running    0     14m
      quay3123hcpcs-quay-mirror-5f68595776-w7dvs   1/1   Terminating  0     56s
      quay3123hcpcs-quay-mirror-5f68595776-z5p2v   1/1   Terminating  0     56s
      quay3123hcpcs-quay-redis-769697698f-kqg4g    1/1   Running    0     65m
      
      oc get hpa
      NAME            REFERENCE               TARGETS                   MINPODS  MAXPODS  REPLICAS  AGE
      quay3123hcpcs-clair-app   Deployment/quay3123hcpcs-clair-app   cpu: <unknown>/90%, memory: <unknown>/90%  3     10    2     66m
      quay3123hcpcs-quay-app   Deployment/quay3123hcpcs-quay-app   cpu: 5%/90%, memory: 39%/90%        5     20    5     66m
      quay3123hcpcs-quay-mirror  Deployment/quay3123hcpcs-quay-mirror  cpu: <unknown>/90%, memory: <unknown>/90%  4     20    2     66m
      
      oc get hpa
      NAME                        REFERENCE                              TARGETS                                     MINPODS   MAXPODS   REPLICAS   AGE
      quay3123hcpcs-clair-app     Deployment/quay3123hcpcs-clair-app     cpu: <unknown>/90%, memory: <unknown>/90%   3         10        2          69m
      quay3123hcpcs-quay-app      Deployment/quay3123hcpcs-quay-app      cpu: 7%/90%, memory: 39%/90%                5         20        5          69m
      quay3123hcpcs-quay-mirror   Deployment/quay3123hcpcs-quay-mirror   cpu: 0%/90%, memory: 27%/90%                4         20        4   

              rh-ee-shudeshp Shubhra Jayant Deshpande
              lzha1981 luffy zhang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: