Major Description:
This is an issue found when pull image from cache, now when the target new organization is configured with low quota(100MB), trying to pull image(600MB) from cache will returned error message "unable to pull image: Error parsing image configuration: Error fetching blob: invalid status code from registry 403 (Forbidden)"
The expected behavior should be letting customers know that the root cause is quota is exceed, give general 403 error code is confusing to Quay Customers.
Quay Image: quay-operator-bundle-container-v3.7.0-73
[root@ip-10-0-1-40 centos]# podman pull quay370.apps.quayperf370.perfscale.devcluster.openshift.com/testpullcache/quay/quay:v3.3.4-2 --tls-verify=false --creds quay:password Trying to pull quay370.apps.quayperf370.perfscale.devcluster.openshift.com/testpullcache/quay/quay:v3.3.4-2... Error fetching blob: invalid status code from registry 403 (Forbidden) Error: error pulling image "quay370.apps.quayperf370.perfscale.devcluster.openshift.com/testpullcache/quay/quay:v3.3.4-2": unable to pull quay370.apps.quayperf370.perfscale.devcluster.openshift.com/testpullcache/quay/quay:v3.3.4-2: unable to pull image: Error parsing image configuration: Error fetching blob: invalid status code from registry 403 (Forbidden)
Quay APP POD Logs:
gunicorn-registry stdout | 2022-04-14 05:43:40,988 [227] [DEBUG] [endpoints.v2] sending response: b'{"errors":[{"code":"DENIED","detail":{},"message":"Quota has been exceeded on namespace"}]}\n' gunicorn-registry stdout | 2022-04-14 05:43:40,989 [227] [DEBUG] [app] Ending request: urn:request:0bdb74b7-22f9-46dd-b1f6-a7eb5caf5e3f (/v2/testpullcache/quay/quay/blobs/sha256:a8e754f8c6e40b5a8534cb4783aaaac6a53b5eab3dab1a254e1096a56573d5be) {'endpoint': 'v2.download_blob', 'request_id': 'urn:request:0bdb74b7-22f9-46dd-b1f6-a7eb5caf5e3f', 'remote_addr': '10.131.0.5', 'http_method': 'GET', 'original_url': 'https://quay370.apps.quayperf370.perfscale.devcluster.openshift.com/v2/testpullcache/quay/quay/blobs/sha256:a8e754f8c6e40b5a8534cb4783aaaac6a53b5eab3dab1a254e1096a56573d5be', 'path': '/v2/testpullcache/quay/quay/blobs/sha256:a8e754f8c6e40b5a8534cb4783aaaac6a53b5eab3dab1a254e1096a56573d5be', 'parameters': {}, 'json_body': None, 'confsha': '529abdae', 'user-agent': 'libpod/1.6.4'} gunicorn-registry stdout | 2022-04-14 05:43:40,990 [227] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-04-14 05:43:40,990 [227] [DEBUG] [peewee.pool] Returning 140244650240752 to pool. gunicorn-registry stdout | 2022-04-14 05:43:40,990 [227] [INFO] [gunicorn.access] 10.131.0.5 - - [14/Apr/2022:05:43:40 +0000] "GET /v2/testpullcache/quay/quay/blobs/sha256:a8e754f8c6e40b5a8534cb4783aaaac6a53b5eab3dab1a254e1096a56573d5be HTTP/1.1" 403 92 "-" "libpod/1.6.4"
