Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-2712

Mirror configuration fails when using Amazon ECR credentials

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • quay-v3.7.0
    • None
    • quay
    • 0

    Description

      When a user requests access credentials for Amazon's ECR, the returned creds are in the form of a JWT token which is 1800+ characters long. After encryption, this token exceeds 2048 characters that the repomirrorconfig table allocates for the password which makes Quay return a 500. This is the trace we see in the logs:

      gunicorn-web stdout | 2021-10-20 07:50:00,757 [218] [ERROR] [gunicorn.error] Error handling request /api/v1/repository/ibazulic/test/mirror
      gunicorn-web stdout | Traceback (most recent call last):
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 55, in handle
      gunicorn-web stdout |     self.handle_request(listener_name, req, client, addr)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/ggevent.py", line 143, in handle_request
      gunicorn-web stdout |     super().handle_request(listener_name, req, sock, addr)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 106, in handle_request
      gunicorn-web stdout |     respiter = self.wsgi(environ, resp.start_response)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2463, in __call__
      gunicorn-web stdout |     return self.wsgi_app(environ, start_response)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/werkzeug/middleware/proxy_fix.py", line 232, in __call__
      gunicorn-web stdout |     return self.app(environ, start_response)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2449, in wsgi_app
      gunicorn-web stdout |     response = self.handle_exception(e)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask_restful/__init__.py", line 269, in error_router
      gunicorn-web stdout |     return original_handler(e)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1866, in handle_exception
      gunicorn-web stdout |     reraise(exc_type, exc_value, tb)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/_compat.py", line 38, in reraise
      gunicorn-web stdout |     raise value.with_traceback(tb)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2446, in wsgi_app
      gunicorn-web stdout |     response = self.full_dispatch_request()
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1951, in full_dispatch_request
      gunicorn-web stdout |     rv = self.handle_user_exception(e)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask_restful/__init__.py", line 269, in error_router
      gunicorn-web stdout |     return original_handler(e)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1820, in handle_user_exception
      gunicorn-web stdout |     reraise(exc_type, exc_value, tb)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/_compat.py", line 38, in reraise
      gunicorn-web stdout |     raise value.with_traceback(tb)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1949, in full_dispatch_request
      gunicorn-web stdout |     rv = self.dispatch_request()
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1935, in dispatch_request
      gunicorn-web stdout |     return self.view_functions[rule.endpoint](**req.view_args)
      gunicorn-web stdout |   File "/quay-registry/endpoints/decorators.py", line 197, in wrapper
      gunicorn-web stdout |     return func(*args, **kwargs)
      gunicorn-web stdout |   File "/quay-registry/auth/decorators.py", line 65, in wrapper
      gunicorn-web stdout |     return func(*args, **kwargs)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask_restful/utils/cors.py", line 35, in wrapped_function
      gunicorn-web stdout |     resp = make_response(f(*args, **kwargs))
      gunicorn-web stdout |   File "/quay-registry/endpoints/csrf.py", line 73, in wrapper
      gunicorn-web stdout |     resp = func(*args, **kwargs)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask_restful/__init__.py", line 458, in wrapper
      gunicorn-web stdout |     resp = resource(*args, **kwargs)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask/views.py", line 89, in view
      gunicorn-web stdout |     return self.dispatch_request(*args, **kwargs)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/flask_restful/__init__.py", line 573, in dispatch_request
      gunicorn-web stdout |     resp = meth(*args, **kwargs)
      gunicorn-web stdout |   File "/quay-registry/endpoints/decorators.py", line 145, in wrapper
      gunicorn-web stdout |     return func(*args, **kwargs)
      gunicorn-web stdout |   File "/quay-registry/endpoints/api/__init__.py", line 241, in wrapper
      gunicorn-web stdout |     return func(namespace, repository, *args, **kwargs)
      gunicorn-web stdout |   File "/quay-registry/endpoints/decorators.py", line 124, in wrapper
      gunicorn-web stdout |     return func(*args, **kwargs)
      gunicorn-web stdout |   File "/quay-registry/endpoints/api/__init__.py", line 295, in wrapped
      gunicorn-web stdout |     return func(self, namespace, repository, *args, **kwargs)
      gunicorn-web stdout |   File "/quay-registry/endpoints/api/__init__.py", line 416, in wrapped
      gunicorn-web stdout |     return func(self, *args, **kwargs)
      gunicorn-web stdout |   File "/quay-registry/endpoints/api/mirror.py", line 402, in put
      gunicorn-web stdout |     if model.repo_mirror.change_credentials(repo, username, password):
      gunicorn-web stdout |   File "/quay-registry/data/model/repo_mirror.py", line 430, in change_credentials
      gunicorn-web stdout |     return bool(update_with_transaction(mirror, **updates))
      gunicorn-web stdout |   File "/quay-registry/data/model/repo_mirror.py", line 355, in update_with_transaction
      gunicorn-web stdout |     if query.execute():
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 1845, in inner
      gunicorn-web stdout |     return method(self, database, *args, **kwargs)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 1916, in execute
      gunicorn-web stdout |     return self._execute(database)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 2400, in _execute
      gunicorn-web stdout |     cursor = database.execute(self)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 3069, in execute
      gunicorn-web stdout |     sql, params = ctx.sql(query).query()
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 606, in sql
      gunicorn-web stdout |     return obj.__sql__(self)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 2453, in __sql__
      gunicorn-web stdout |     (ctx
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 606, in sql
      gunicorn-web stdout |     return obj.__sql__(self)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 1709, in __sql__
      gunicorn-web stdout |     ctx.sql(self.nodes[i])
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 606, in sql
      gunicorn-web stdout |     return obj.__sql__(self)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 1711, in __sql__
      gunicorn-web stdout |     ctx.sql(self.nodes[n_nodes - 1])
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 606, in sql
      gunicorn-web stdout |     return obj.__sql__(self)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 1338, in __sql__
      gunicorn-web stdout |     return ctx.value(self.value, self.converter)
      gunicorn-web stdout |   File "/usr/local/lib/python3.8/site-packages/peewee.py", line 618, in value
      gunicorn-web stdout |     value = converter(value)
      gunicorn-web stdout |   File "/quay-registry/data/fields.py", line 176, in db_value
      gunicorn-web stdout |     return meta.encrypter.encrypt_value(
      gunicorn-web stdout |   File "/quay-registry/data/encryption.py", line 78, in encrypt_value
      gunicorn-web stdout |     encrypted_value = self._encryption_version.encrypt(
      gunicorn-web stdout |   File "/quay-registry/data/encryption.py", line 33, in _encrypt_ccm
      gunicorn-web stdout |     assert (len(encrypted) + _RESERVED_FIELD_SPACE) <= field_max_length, msg
      gunicorn-web stdout | AssertionError: Tried to encode a value too large for this field
      gunicorn-web stdout | 2021-10-20 07:50:00,758 [218] [INFO] [gunicorn.access]  - - [20/Oct/2021:07:50:00 +0000] "PUT /api/v1/repository/ibazulic/test/mirror HTTP/1.0" 500 0 "-
      nginx stdout | 10.100.102.3 () - - [20/Oct/2021:07:50:00 +0000] "PUT /api/v1/repository/ibazulic/test/mirror HTTP/2.0" 500 141 "https://enterprisequay.rblbank.com/repository/illa/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36" (0.052 2848 0.052)
      

      Please check!

      Attachments

        Activity

          People

            sleesinc Kenny Lee Sin Cheong
            rhn-support-ibazulic Ivan Bazulic
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: