Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-1282

Quay Click Canceled build Job hit quay 500 error page

XMLWordPrintable

      Description:

      This is an issue found when canceled build job, after create Github Build trigger(Repository Push) Job,  submit commit to target Git repository, then new Build was generated, the status is "Waiting for available build worker", canceled this job. Come back to the build page, click the canceled Job, hit quay 500 error page, see Quay POD logs.

      Quay Image:

      lizhang@lzha-mac quay3.4 % oc get pod quaybuilder-quay-app-565cf8c5f5-7w9m2 -o json | jq '.spec.containers[0].image'
      "registry.redhat.io/quay/quay@sha256:12434420fa3ca1df4654bf6d08c8980b1209b13fbfe8336fd9b1ef6f04cc4c75"
      

       

      Quay POD Logs:

      unicorn-web stdout | 2020-11-13 06:38:30,329 [299] [ERROR] [gunicorn.error] Error handling request /api/v1/repository/quay/demo/build/0e216fd9-8510-4d56-98ca-c950acebd7b7/logs?start=0
      gunicorn-web stdout | Traceback (most recent call last):
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 55, in handle
      gunicorn-web stdout |     self.handle_request(listener_name, req, client, addr)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/ggevent.py", line 143, in handle_request
      gunicorn-web stdout |     super().handle_request(listener_name, req, sock, addr)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 106, in handle_request
      gunicorn-web stdout |     respiter = self.wsgi(environ, resp.start_response)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2463, in __call__
      gunicorn-web stdout |     return self.wsgi_app(environ, start_response)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/werkzeug/middleware/proxy_fix.py", line 232, in __call__
      gunicorn-web stdout |     return self.app(environ, start_response)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2449, in wsgi_app
      gunicorn-web stdout |     response = self.handle_exception(e)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask_restful/__init__.py", line 269, in error_router
      gunicorn-web stdout |     return original_handler(e)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1866, in handle_exception
      gunicorn-web stdout |     reraise(exc_type, exc_value, tb)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/_compat.py", line 38, in reraise
      gunicorn-web stdout |     raise value.with_traceback(tb)
      ......
      gunicorn-web stdout |     return func(self, namespace, repository, *args, **kwargs)
      gunicorn-web stdout |   File "/quay-registry/endpoints/api/__init__.py", line 266, in wrapped
      gunicorn-web stdout |     return func(self, namespace_name, repository_name, *args, **kwargs)
      gunicorn-web stdout |   File "/quay-registry/endpoints/api/build.py", line 513, in get
      gunicorn-web stdout |     return get_logs_or_log_url(build)
      gunicorn-web stdout |   File "/quay-registry/endpoints/api/build.py", line 469, in get_logs_or_log_url
      gunicorn-web stdout |     "logs_url": log_archive.get_file_url(build.uuid, get_request_ip(), requires_cors=True)
      gunicorn-web stdout |   File "/quay-registry/data/userfiles.py", line 121, in get_file_url
      gunicorn-web stdout |     url = self._storage.get_direct_download_url(
      gunicorn-web stdout |   File "/quay-registry/storage/distributedstorage.py", line 82, in get_direct_download_url
      gunicorn-web stdout |     download_url = self._get_direct_download_url(
      gunicorn-web stdout |   File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper
      gunicorn-web stdout |     storage = self._storages[random.sample(locations, 1)[0]]
      gunicorn-web stdout | KeyError: 'local_us'
      gunicorn-web stdout | 2020-11-13 06:38:30,332 [299] [INFO] [gunicorn.access]  - - [13/Nov/2020:06:38:30 +0000] "GET /api/v1/repository/quay/demo/build/0e216fd9-8510-4d56-98ca-c950acebd7b7/logs?start=0 HTTP/1.0" 500 0 "-" "-"
      

      Quay config.yaml:

      ALLOW_PULLS_WITHOUT_STRICT_LOGGING: false
      AUTHENTICATION_TYPE: Database
      BUILDLOGS_REDIS:
        host: quaybuilder-quay-redis
        port: 6379
      DATABASE_SECRET_KEY: 1AC0q8i4voTx2vb3-XfVkFGTJbSENTJlfqonX1EeyUnsL6ROHagGDfB2WGha00NPUmkqc333lpBSUdlV
      DB_CONNECTION_ARGS:
        autorollback: true
        threadlocals: true
      DB_URI: postgresql://quaybuilder-quay-database:postgres@quaybuilder-quay-database:5432/quaybuilder-quay-database
      DEFAULT_TAG_EXPIRATION: 2w
      DISTRIBUTED_STORAGE_CONFIG:
        default:
        - S3Storage
        - host: s3.us-east-2.amazonaws.com
          port: "443"
          s3_access_key: *****
          s3_bucket: quay340
          s3_secret_key: *****
          storage_path: /quay3401113
      DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS:
      - default
      DISTRIBUTED_STORAGE_PREFERENCE:
      - default
      ENTERPRISE_LOGO_URL: /static/img/quay-horizontal-color.svg
      FEATURE_BUILD_SUPPORT: true
      FEATURE_DIRECT_LOGIN: true
      FEATURE_GITHUB_BUILD: true
      FEATURE_MAILING: false
      FEATURE_REPO_MIRROR: true
      FEATURE_SECURITY_NOTIFICATIONS: true
      FEATURE_SECURITY_SCANNER: true
      GITHUB_TRIGGER_CONFIG:
        API_ENDPOINT: https://api.github.com/
        CLIENT_ID: ****
        CLIENT_SECRET: *****
        GITHUB_ENDPOINT: https://github.com/
      PREFERRED_URL_SCHEME: https
      REGISTRY_TITLE: Quay
      REGISTRY_TITLE_SHORT: Quay
      REPO_MIRROR_INTERVAL: 30
      REPO_MIRROR_TLS_VERIFY: true
      SECRET_KEY: M-b4mukLnJGoD6PjzQuBsNjYar0cQCJylvK5NKXaubQtixStW3JnKMYSvYeWeBy7gRi1fhC2vjwhgY2V
      SECURITY_SCANNER_INDEXING_INTERVAL: 30
      SECURITY_SCANNER_V4_ENDPOINT: http://quaybuilder-clair:80
      SECURITY_SCANNER_V4_NAMESPACE_WHITELIST:
      - admin
      SECURITY_SCANNER_V4_PSK: RHZvMkZma1hjS1lpNjVIV0hPS2dTczVlRTA0Y0plNTc=
      SERVER_HOSTNAME: quaybuilder-quay-quaybuilder.apps.quay-perf-399.perf-testing.devcluster.openshift.com
      SETUP_COMPLETE: true
      SUPER_USERS:
      - quay
      - admin
      TAG_EXPIRATION_OPTIONS:
      - 2w
      TEAM_RESYNC_STALE_TIME: 60m
      TESTING: false
      USER_EVENTS_REDIS:
        host: quaybuilder-quay-redis
        port: 6379
      

      Steps:

      1. Deploy Quay with TNG Operator with including GitHub Trigger Config settings
      2. Login Quay
      3. Create new image repository
      4. Create new Github build trigger, fill all required settings
      5. Submit commit to the Github repository
      6. Wait to see new build generated
      7. Cancel the build
      8. Navigate to build page, click the canceled build Job(see screenshot)

      Expected Results:

      Canceled Job can be displayed correctly.

      Actual Results:

      Hit Quay 500 error page.

        1. quay340.logs
          520 kB
          luffy zhang
        2. image-2020-12-17-17-40-01-892.png
          198 kB
          luffy zhang
        3. Click canceled build 0e216fd9.png
          517 kB
          luffy zhang

              sleesinc Kenny Lee Sin Cheong
              lzha1981 luffy zhang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: