Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-2135

Quay upgrade pods running all workers instead of just database upgrade

    XMLWordPrintable

Details

    • 0

    Description

      We saw that Quay upgrade pods start all workers and not just database migrations. In a client's case, this had an unfortunate sideeffect where all images were sent to be scanned by Clair v4 instead of just exiting once alembic operations are done. They were testing the upgrade so they used their production environment but without hooking up storage so the security worker ended in a loop where it repeatedly enqueued same layers over and over for scanning and the pod never progressed. This was resolved by deleting the upgrade pods and letting them spin again at which point a correct db version was detected and the instalaltion progressed normally.

      Expected outcome: we only run alembic operations in the upgrade pod without any other workers and then progress and run the main Quay app. The main app should handle all other layer manipulation.

      Attachments

        Issue Links

          Activity

            People

              rhn-coreos-amerdler Alec Merdler (Inactive)
              rhn-support-ibazulic Ivan Bazulic
              Dongbo Yan Dongbo Yan
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: