Loading...

Type: Bug
Resolution: Done
Priority: Blocker
Fix Version/s: None
Affects Version/s: quay-v3.14.1
Component/s: quay-operator
Labels:
- triaged

Blocked:
False
Blocked Reason:

Hide

None

Show
None
Ready:
False

Release Blocker:
Approved
Target Version:

quay-v3.9.0

SFDC Cases Links:
SFDC Cases Open:
SFDC Cases Counter:

Description:

When Quay registry start to reconcile, related secret bundle was removed, which cause pods can't startup

Index Image:

quay-operator-bundle-container-v3.14.1-9

Steps:

1, Create secret bundle, can see it created successfully, and it can be found on OCP Secrets page

% oc create secret generic --from-file config.yaml=./config.yaml test-config-bundle 
secret/test-config-bundle created

2, Deploy quay registry with the secret

Result: the secret was removed

 % oc get quayregistry quayregistry -oyaml
apiVersion: quay.redhat.com/v1
kind: QuayRegistry
metadata:
  creationTimestamp: "2025-05-07T01:55:17Z"
  generation: 2
  name: quayregistry
  namespace: quay-enterprise-15270
  resourceVersion: "44309"
  uid: 8e950477-f10d-402a-8dd0-a266f08cab33
spec:
  components:
  - kind: tls
    managed: true
  - kind: quay
    managed: true
    overrides:
      env:
      - name: DEBUGLOG
        value: "true"
  - kind: postgres
    managed: true
  - kind: clair
    managed: true
  - kind: redis
    managed: true
  - kind: horizontalpodautoscaler
    managed: true
  - kind: objectstorage
    managed: true
  - kind: route
    managed: true
  - kind: mirror
    managed: true
  - kind: monitoring
    managed: false
  - kind: clairpostgres
    managed: true
  configBundleSecret: test-config-bundle
status:
  conditions:
  - lastTransitionTime: "2025-05-07T01:55:22Z"
    lastUpdateTime: "2025-05-07T01:55:48Z"
    message: Horizontal pod autoscaler found
    reason: ComponentReady
    status: "True"
    type: ComponentHPAReady
  - lastTransitionTime: "2025-05-07T01:55:22Z"
    lastUpdateTime: "2025-05-07T01:55:48Z"
    message: Route admitted
    reason: ComponentReady
    status: "True"
    type: ComponentRouteReady
  - lastTransitionTime: "2025-05-07T01:55:17Z"
    lastUpdateTime: "2025-05-07T01:55:48Z"
    message: Monitoring not managed by the operator
    reason: ComponentNotManaged
    status: "True"
    type: ComponentMonitoringReady
  - lastTransitionTime: "2025-05-07T01:55:48Z"
    lastUpdateTime: "2025-05-07T01:55:48Z"
    message: Deployment quayregistry-quay-database healthy
    reason: ComponentReady
    status: "True"
    type: ComponentPostgresReady
  - lastTransitionTime: "2025-05-07T01:55:22Z"
    lastUpdateTime: "2025-05-07T01:55:48Z"
    message: Object bucket claim bound
    reason: ComponentReady
    status: "True"
    type: ComponentObjectStorageReady
  - lastTransitionTime: "2025-05-07T01:55:18Z"
    lastUpdateTime: "2025-05-07T01:55:48Z"
    message: Deployment quayregistry-clair-app has zero replicas available
    reason: ComponentNotReady
    status: "False"
    type: ComponentClairReady
  - lastTransitionTime: "2025-05-07T01:55:48Z"
    lastUpdateTime: "2025-05-07T01:55:48Z"
    message: ClairPostgres component healthy
    reason: ComponentReady
    status: "True"
    type: ComponentClairPostgresReady
  - lastTransitionTime: "2025-05-07T01:55:22Z"
    lastUpdateTime: "2025-05-07T01:55:48Z"
    message: Config bundle does not exist
    reason: ComponentNotReady
    status: "False"
    type: ComponentTLSReady
  - lastTransitionTime: "2025-05-07T01:55:38Z"
    lastUpdateTime: "2025-05-07T01:55:48Z"
    message: Deployment quayregistry-quay-redis healthy
    reason: ComponentReady
    status: "True"
    type: ComponentRedisReady
  - lastTransitionTime: "2025-05-07T01:55:17Z"
    lastUpdateTime: "2025-05-07T01:55:48Z"
    message: Awaiting for component clair,tls to become available
    reason: ComponentNotReady
    status: "False"
    type: ComponentQuayReady
  - lastUpdateTime: "2025-05-07T01:55:48Z"
    message: Awaiting for component quay to become available
    reason: ComponentNotReady
    status: "False"
    type: ComponentMirrorReady
  - lastTransitionTime: "2025-05-07T01:55:17Z"
    lastUpdateTime: "2025-05-07T01:55:48Z"
    message: Some components are not ready
    reason: ComponentNotReady
    status: "False"
    type: Available
  - lastTransitionTime: "2025-05-07T01:55:48Z"
    lastUpdateTime: "2025-05-07T01:55:48Z"
    message: 'unable to get `configBundleSecret`: Secret "test-config-bundle" not
      found'
    reason: ConfigInvalid
    status: "True"
    type: RolloutBlocked
  lastUpdated: 2025-05-07 01:55:48.622332168 +0000 UTC

% oc get pod
NAME                                          READY   STATUS                  RESTARTS        AGE
quay-operator.v3.14.1-7898b6c5cb-2sbb2        1/1     Running                 0               38m
quayregistry-clair-app-77fd576bbb-5vqpv       1/1     Running                 4 (17m ago)     18m
quayregistry-clair-app-77fd576bbb-whhrs       1/1     Running                 0               16m
quayregistry-clair-postgres-f5d6dcb55-r9hk9   1/1     Running                 0               18m
quayregistry-quay-app-764bb7b7fd-4wfqj        1/1     Running                 0               3m5s
quayregistry-quay-app-764bb7b7fd-q6x6h        1/1     Running                 0               3m11s
quayregistry-quay-app-upgrade-dh4tw           0/1     Completed               0               18m
quayregistry-quay-database-79764765f4-b7sqt   1/1     Running                 0               18m
quayregistry-quay-mirror-5b94f66fc-6sljd      0/1     Init:CrashLoopBackOff   8 (2m32s ago)   18m
quayregistry-quay-mirror-5b94f66fc-bmtgq      0/1     Init:CrashLoopBackOff   8 (2m56s ago)   18m
quayregistry-quay-redis-6746bf4c9c-nm56w      1/1     Running                 0               18m

This doesn't related with config.yaml, it looks always happen.

Delete test-config-bundle, and create a new one?

{"level":"info","ts":"2025-05-07T01:55:20Z","logger":"controllers.QuayRegistry","msg":"deleting old objects","quayregistry":"quay-enterprise-15270/quayregistry"}
{"level":"info","ts":"2025-05-07T01:55:20Z","logger":"controllers.QuayRegistry","msg":"creating/updating object","quayregistry":"quay-enterprise-15270/quayregistry","kind":"Secret","name":"quayregistry-quay-config-secret-9g8775mb7g"}