Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-10068

Proxying of manifests containing multiple same child manifests fails on Quay

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • quay-v3.15.0
    • quay
    • False
    • Hide

      None

      Show
      None
    • False
    • Quay Enterprise

      If an original manifest has multiple identical entries (same child manifests, same digest, same arch), then proxying of such a manifest throgh Quay's proxy function will fail during manifest child insertions. OCI model does not say that a fat manifest (an image index) must have unique entries. Example:

      ghcr.io/berriai/litellm-non_root:litellm_rc_branch-v1.80.9.dev5
      

      This image has 4 child manifests, each appearing twice:

      # skopeo inspect --raw docker://ghcr.io/berriai/litellm-non_root:litellm_rc_branch-v1.80.9.dev5 | jq '.manifests[].digest' | sort | uniq -c
            2 "sha256:082e5af6d1b1e58521fa97f8f212a189cd415103db1936dd57c0b2e8dfa7dad8"
            2 "sha256:4da897392752d9b624f0bccac293ed8b2567fcc5c3be56a917a40d5b57d01b55"
            2 "sha256:9ea73ef1405bf04ec737edd37b6c7dda2bdbbab262804f9babfc34f33e9c090e"
            2 "sha256:f64613434b1e79f7ae3e9b840c5592305c0e01a1f5b83ba27e2d752ce61cd2fd"
      
      # skopeo inspect --raw docker://ghcr.io/berriai/litellm-non_root:litellm_rc_branch-v1.80.9.dev5 | jq '.manifests[] | select(.platform.architecture == "amd64")'
      {
        "mediaType": "application/vnd.oci.image.manifest.v1+json",
        "digest": "sha256:f64613434b1e79f7ae3e9b840c5592305c0e01a1f5b83ba27e2d752ce61cd2fd",
        "size": 5629,
        "platform": {
          "architecture": "amd64",
          "os": "linux"
        }
      }
      {
        "mediaType": "application/vnd.oci.image.manifest.v1+json",
        "digest": "sha256:f64613434b1e79f7ae3e9b840c5592305c0e01a1f5b83ba27e2d752ce61cd2fd",
        "size": 5629,
        "platform": {
          "architecture": "amd64",
          "os": "linux"
        }
      }
      

      When proxying this manifest through Quay, the following INSERT fails:

      gunicorn-registry stdout | 2025-12-18 07:48:06,052 [356] [DEBUG] [peewee] ('INSERT INTO "manifestchild" ("repository_id", "manifest_id", "child_manifest_id") VALUES (%s, %s, %s), (%s, %s, %s), (%s, %s, %s), (%s, %s, %s), (%s, %s, %s), (%s, %s, %s), (%s, %s, %s), (%s, %s, %s) RETURNING "manifestchild"."id"', [3, 10, 11, 3, 10, 11, 3, 10, 12, 3, 10, 12, 3, 10, 13, 3, 10, 13, 3, 10, 14, 3, 10, 14])
      

      As you can see, each set of 3 numbers in the insert is duplicated:

      2x (3, 10, 11)
      2x (3, 10, 12)
      2x (3, 10, 13)
      2x (3, 10, 14)
      

      This causes the unique constraint violation exception to be raised causing a failure of the pull:

      gunicorn-registry stdout | 2025-12-18 07:48:06,052 [356] [ERROR] [gunicorn.error] Error handling request /v2/ghcr-proxy/ghcr.io/berriai/litellm-non_root/manifests/main-v1.80.10.rc.4
      gunicorn-registry stdout | Traceback (most recent call last):
      gunicorn-registry stdout |   File "/app/lib/python3.12/site-packages/peewee.py", line 3057, in execute_sql
      gunicorn-registry stdout |     cursor.execute(sql, params or ())
      gunicorn-registry stdout | psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "manifestchild_manifest_id_child_manifest_id"
      gunicorn-registry stdout | DETAIL:  Key (manifest_id, child_manifest_id)=(10, 11) already exists.
      

      Copying the same image with skopeo or pushing it with podman to Quay works as expected, the image can be copied over and no errors are raised.

      Please check, thanks!

              Unassigned Unassigned
              rhn-support-ibazulic Ivan Bazulic
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: