Story: As a Quay administrator I expect the Operator to constantly reconcile all managed components and their related resources so that failed deployment / reconciliation attempts are retried and it continues deploying / running / updating my Quay deployment after I fixed a missing dependencies, misconfiguration or manual resource modification and removal.
Background / Reproducer: Attempt to install a QuayRegistry with objectstorage as managed without deploying NooBaa first. First, the missing ObjectBucketClaim API error can only be seen in the Operator pod logs (targeted fix via
PROJQUAY-1609). After install of that Operator the Quay deployment does not proceed with a new attempt at all but it should. Removing and re-applying the same QuayRegistry object fails RolloutBlocked because of ComponentCreationFailed. The root cause is that the user forgot to place the NooBaa CR and initialize the local object storage gateway. After placing the NooBaa CR the Quay Operator does not attempt to reconcile the QuayRegistry CR again but it should.
Before we implement this behavior we need to eliminate all causes for users to adjust the configuration of deployed components manually and thus by-passing the operator (see dependencies)
- Clair and Quay can get their proxy information as part of the config files
- The minimum replica amount for all scale-out components needs to be 2
- All scale-out components need to have proper scheduling constraints set to spread across nodes and zones
- Clair and Quay need to be able to get their debug log level set as part of the config files
- the Quay Operator continuously tries to reconcile periodically also in the event of errors
- quay-app upgrade jobs runs only when it is needed (version mismatch)
- go routine that monitors the state of the migration job does not exist anymore