Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-824

BuildKit's --export-cache function does not work on Quay.io

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Minor
    • 2020.08.20
    • None
    • quay, quay.io
    • Quay Enterprise
    • 0

    Description

      Zendesk ticket: https://coreos.zendesk.com/agent/tickets/18495

      Client can't push images built with BuildKit to Quay.io, Quay returns a 400 - Bad request. I have replicated the issue on my own Quay deployment and the following can be seen in Quay's log:

      gunicorn-registry stdout | 2020-07-01 13:32:07,273 [713] [ERROR] [endpoints.v2.manifest] failed to parse manifest when writing by tagname Traceback (most recent call last): File "/quay-registry/endpoints/v2/manifest.py", line 274, in _parse_manifest return parse_manifest_from_bytes(Bytes.for_string_or_unicode(request.data), content_type) File "/quay-registry/image/shared/schemas.py", line 35, in parse_manifest_from_bytes return DockerSchema2ManifestList(manifest_bytes) File "/quay-registry/image/docker/schema2/list.py", line 170, in __init__ raise MalformedSchema2ManifestList("manifest data does not match schema: %s" % ve) MalformedSchema2ManifestList: manifest data does not match schema: 'platform' is a required property
      

      I proceeded to dump the created image to a local folder and found two manifests for cache. First one looks normal, it just references the config blob from the image:

      {
        "schemaVersion": 2,
        "manifests": [
          {
            "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
            "digest": "sha256:95250204165b0f1ca21bd476a24431fa5d00b1079a9a5b7be08ef117d4a862e0",
            "size": 1284,
            "annotations": {
              "org.opencontainers.image.ref.name": "latest"
            }
          }
        ]
      }

      This is the config blob referenced by this manifest:

      {
        "schemaVersion": 2,
        "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
        "manifests": [
          {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "digest": "sha256:6910e5a164f725142d77994b247ba20040477fbab49a721bdbe8d61cf855ac23",
            "size": 74866818,
            "annotations": {
              "buildkit/createdat": "2020-07-01T15:31:41.422256279+02:00",
              "containerd.io/uncompressed": "sha256:eb29745b8228e1e97c01b1d5c2554a319c00a94d8dd5746a3904222ad65a13f8"
            }
          },
          {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "digest": "sha256:7e5d6a68556dd3418031fcae5b97f96694c71f704c10db36c545d0756a2fb1a1",
            "size": 22126432,
            "annotations": {
              "buildkit/createdat": "2020-07-01T15:31:52.160208976+02:00",
              "containerd.io/uncompressed": "sha256:bea83166e742e283c12916e273e9c375de245e452294fbbdefe74e1b864959bf"
            }
          },
          {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "digest": "sha256:eb320cb0894907ec2e4bf27ca8e64602353da602e2aadd2914cd31ff1c0bac36",
            "size": 19868697,
            "annotations": {
              "buildkit/createdat": "2020-07-01T15:31:47.596567113+02:00",
              "containerd.io/uncompressed": "sha256:06fa8d6a982444c9f11f6f8955bdbbe638284f279a7a67d2ca40207f39d64316"
            }
          },
          {
            "mediaType": "application/vnd.buildkit.cacheconfig.v0",
            "digest": "sha256:51dfd66ed5abd8d2000b5b2f711abb6ffe3d39984b9d868357df2d5795e0564a",
            "size": 841
          }
        ]
      }
      

      The last blob is an application/vnd.buildkit.cacheconfig.v0 which is not in the list of supported media types for Quay, which is why I'm guessing that the error returned by Quay is in fact a red herring. The client also opened a bug report on Moby, ref: https://github.com/moby/buildkit/issues/1550

      According to them, platform is not a required property, so Quay should not fail on that particular error, but fail on the unknown blob type, in my opinion.

      Can you please verify and confirm? Thanks!

      Attachments

        Activity

          People

            Unassigned Unassigned
            rhn-support-ibazulic Ivan Bazulic
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: