Details
-
Bug
-
Resolution: Done
-
Major
-
quay-v3.3.1
Description
When FEATURE_ANONYMOUS_ACCESS is set to false in Quay's config.yaml file, CSO will fail to load security information even though credentials are correct. This is the error seen in Quay's log:
gunicorn-web stdout | 2020-10-09 10:01:58,152 [500] [DEBUG] [app] Starting request: urn:request:87b7a868-6385-434b-ac10-fdede843cf04 (/.well-known/app-c apabilities) gunicorn-web stdout | 2020-10-09 10:01:58,153 [500] [DEBUG] [app] Ending request: urn:request:87b7a868-6385-434b-ac10-fdede843cf04 (/.well-known/app-cap abilities) nginx stdout | 10.128.2.7 () - - [09/Oct/2020:10:01:58 +0000] "GET /.well-known/app-capabilities HTTP/1.1" 200 490 "-" "Go-http-client/1.1" (0.002 139 0 .002) gunicorn-web stdout | 2020-10-09 10:01:58,153 [500] [INFO] [gunicorn.access] 10.128.2.7 - - [09/Oct/2020:10:01:58 +0000] "GET /.well-known/app-capabilit ies HTTP/1.0" 200 490 "-" "Go-http-client/1.1" gunicorn-web stdout | 2020-10-09 10:01:58,157 [509] [DEBUG] [app] Starting request: urn:request:c99e5121-22ae-4121-bed3-6214d61f184e (/api/v1/repository/ibazulic/debian-build-test/manifest/sha256:46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6/security) gunicorn-web stdout | 2020-10-09 10:01:58,157 [509] [DEBUG] [auth.oauth] Got invalid bearer token format: Basic aWJhenVsaWM6dGVzdDEyMzQ= gunicorn-web stdout | 2020-10-09 10:01:58,158 [509] [ERROR] [util.http] Error 401: Anonymous access is not allowed; Arguments: {'url': u'https://quay.apps.ibazulic.ibazulic.me/api/v1/repository/ibazulic/debian-build-test/manifest/sha256:46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6/security?features=true&vulnerabilities=true', 'status_code': 401, 'message': 'Anonymous access is not allowed', 'manifestref': u'sha256:46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6', 'repository': u'ibazulic/debian-build-test'} gunicorn-web stdout | 2020-10-09 10:01:58,160 [509] [DEBUG] [app] Ending request: urn:request:c99e5121-22ae-4121-bed3-6214d61f184e (/api/v1/repository/ibazulic/debian-build-test/manifest/sha256:46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6/security) nginx stdout | 10.128.2.7 () - ibazulic [09/Oct/2020:10:01:58 +0000] "GET /api/v1/repository/ibazulic/debian-build-test/manifest/sha256:46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6/security?features=true&vulnerabilities=true HTTP/1.1" 401 47 "-" "Go-http-client/1.1" (0.006 327 0.006) gunicorn-web stdout | 2020-10-09 10:01:58,161 [509] [INFO] [gunicorn.access] 10.128.2.7 - ibazulic [09/Oct/2020:10:01:58 +0000] "GET /api/v1/repository/ibazulic/debian-build-test/manifest/sha256:46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6/security?features=true&vulnerabilities=true HTTP/1.0" 401 47 "-" "Go-http-client/1.1"
CSO shows the following:
level=debug msg="Pod updated" key=test-cso-operator/test-cso-operator level=info msg="Garbage collecting unreferenced ImageManifestVulns" key=test-cso-operator/test-cso-operator level=error msg="Failed to sync layer data" key=test-cso-operator/test-cso-operator err="Request returned non-200 response: 401 UNAUTHORIZED"
When FEATURE_ANONYMOUS_ACCESS is set to true, access is granted to CSO:
level=info msg="Created ImageManifestVuln" manifestKey=test-cso-operator/sha256.46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6 key=test-cso-operator/test-cso-operator level=debug msg="ImageManifestVuln added" key=test-cso-operator/sha256.46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6 level=debug msg="ImageManifestVuln updated" key=test-cso-operator/sha256.46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6
And Quay logs show the normal auth flow for basic auth:
gunicorn-web stdout | 2020-10-09 10:13:11,584 [529] [DEBUG] [app] Starting request: urn:request:db5fe139-90f3-4b54-8f06-d5ea3f8e4bd9 (/api/v1/repository/ibazulic/debian-build-test/manifest/sha256:46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6/security) gunicorn-web stdout | 2020-10-09 10:13:11,585 [529] [DEBUG] [auth.oauth] Got invalid bearer token format: Basic aWJhenVsaWM6dGVzdDEyMzQ= gunicorn-web stdout | 2020-10-09 10:13:11,586 [529] [DEBUG] [auth.basic] Attempt to process basic auth header --- gunicorn-web stdout | 2020-10-09 10:13:11,934 [529] [DEBUG] [util.secscan.api] GETing security URL https://quay-clair.quay-enterprise.svc:6060/v1/layers/ccde1ab5772d24a04abbb53b81d1474d6f0ac573d2e5a9c28b749952e52c2fec.b0fc4b57-b977-45ab-b30a-e89813b26b5a gunicorn-web stdout | 2020-10-09 10:13:11,938 [529] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quay-clair.quay-enterprise.svc:6060 --- gunicorn-web stdout | 2020-10-09 10:13:12,095 [529] [DEBUG] [app] Ending request: urn:request:db5fe139-90f3-4b54-8f06-d5ea3f8e4bd9 (/api/v1/repository/ibazulic/debian-build-test/manifest/sha256:46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6/security) gunicorn-web stdout | 2020-10-09 10:13:12,095 [529] [INFO] [gunicorn.access] 10.128.2.7 - ibazulic [09/Oct/2020:10:13:12 +0000] "GET /api/v1/repository/ibazulic/debian-build-test/manifest/sha256:46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6/security?features=true&vulnerabilities=true HTTP/1.0" 200 139602 "-" "Go-http-client/1.1" nginx stdout | 10.128.2.7 () - ibazulic [09/Oct/2020:10:13:12 +0000] "GET /api/v1/repository/ibazulic/debian-build-test/manifest/sha256:46616b5007e107110c138d7ae24bbb2ddb8c3f779d95660b0d76d0440e8821b6/security?features=true&vulnerabilities=true HTTP/1.1" 200 139602 "-" "Go-http-client/1.1" (0.512 327 0.512)
This is similar to the bug https://issues.redhat.com/browse/QUAY-2054 but in that case the user was using local storage. In my and customer's case, we're using cloud based storage. This is entirely connected to the way auth flow for this API endpoint works.
Can you please check? Thank you!