Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-4356

Quay 3.7 operator can't reconcile the change after upload correct TLS Cert for Mysql DB via config editor

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Do
    • Critical
    • None
    • quay-v3.7.7
    • quay-operator
    • False
    • None
    • False
    • 0

    Description

      Description:

      This is an issue found when try to config Quay to use Azure Mysql DB with forced TLS Connection, at the 1st the time upload wrong TLS Cert, then the Quay APP Pod was failed to start, later upload correct TLS Cert again with Config editor, found Quay Operator can't reconcile the change.

      Quay Image: Quay 3.7.7

      oc get pod
      NAME                                              READY   STATUS             RESTARTS        AGE
      quay-operator.v3.7.7-775c5fb8db-nrhzc             1/1     Running            0               3h3m
      quayregistry-clair-app-bd9df7d67-7wsn8            1/1     Running            0               9m17s
      quayregistry-clair-app-bd9df7d67-zl2xr            1/1     Running            0               9m6s
      quayregistry-clair-postgres-56f4957db-wqwxr       1/1     Running            1 (179m ago)    3h
      quayregistry-quay-app-5ccc758d57-6fdmc            0/1     CrashLoopBackOff   6 (2m47s ago)   9m9s
      quayregistry-quay-app-5ccc758d57-glshw            0/1     CrashLoopBackOff   6 (2m28s ago)   8m58s
      quayregistry-quay-app-64ff57dd5c-wt26d            0/1     CrashLoopBackOff   6 (16s ago)     6m42s
      quayregistry-quay-config-editor-dfdb79bf6-qfvjr   1/1     Running            0               6m42s
      quayregistry-quay-database-b6f45dc9c-6dh9h        1/1     Running            0               3h
      quayregistry-quay-mirror-5dc56d7646-drnt6         0/1     Init:0/1           2 (2m7s ago)    6m32s
      quayregistry-quay-mirror-5dc56d7646-pkzv8         0/1     Init:0/1           2 (2m6s ago)    6m32s
      quayregistry-quay-redis-6d85c8cbd8-2k65w          1/1     Running            0               3h  
      
      sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'quay3800.mysql.database.azure.com' ([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131))") (Background on this error at: https://sqlalche.me/e/14/e3q8)
      

       

      Steps:

      1. Deploy Quay with Quay Operator, choose to use managed postgresql DB
      2. Login Quay Config editor to config to use Azure Mysql DB with forced TLS Connection, but upload wrong TLS CA Cert, click reconfigure quay
      3. Quay Operator reconcile the change, but quay upgrade job was failed with error message "sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'quay3800.mysql.database.azure.com' ([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131))")
        *(Background on this error at: https://sqlalche.me/e/14/e3q8)*"
      4. Login Quay config editor, upload correct TLS CA Cert of mysql db, and click reconfigure quay
      5. Wait for Quay Operator to reconcile the change

      Expected Result:

      Quay Operator should be able to reconcile the change, and new Quay upgrade job was triggered to create schema objects on target Azure Mysql DB.

      Actual Results:

      Quay Operator can't reconcile the change.

      Attachments

        Activity

          People

            jonathankingfc Jonathan King
            lzha1981 luffy zhang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: