Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-7671

Quay 3.12.1 push image was failed when use Hitachi HCP Storage

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • quay-v3.12.3
    • quay-v3.12.1
    • quay
    • False
    • None
    • False

      Description:

      This is an issue found in Quay 3.12.1, with new config "signature_version: v4", found when Quay configured with Hitachi HCP V9.7, frequently hit error during push image to Quay, pls review this issue.

      Quay: 3.12.1 quay-operator-bundle-container-v3.12.1-13

      podman tag docker.io/library/python quay3121hitachi-quay-quay3121hcpv97.apps.quaytest-2954.qe.devcluster.openshift.com/quay/python3

podman push quay3121hitachi-quay-quay3121hcpv97.apps.quaytest-2954.qe.devcluster.openshift.com/quay/python3 --tls-verify=f
alse --creds quay:******
Getting image source signatures
Copying blob 916d866d5b0d done   | 
Copying blob 0d80db6a0977 done   | 
Copying blob 418eda943d8d done   | 
Copying blob 28e03088bc15 done   | 
Copying blob 2c3d7750fa05 done   | 
Copying blob 8f4ceb8cc1a2 done   | 
Copying blob 06a1f3247240 done   | 
Copying blob 5f7a708450e3 done   | 
WARN[0070] Failed, retrying in 1s ... (1/3). Error: writing blob: Patch "https://quay3121hitachi-quay-quay3121hcpv97.apps.quaytest-2954.qe.devcluster.openshift.com/v2/quay/python3/blobs/uploads/4be559d8-19ff-4049-8a3f-539d06b730cb": EOF 
Getting image source signatures
Copying blob 4d9811414818 skipped: already exists  
Copying blob baeb38530ae9 skipped: already exists  
Copying blob 076d341e6674 skipped: already exists  
Copying blob 4af5b82c5321 skipped: already exists  
Copying blob 28e03088bc15 done   | 
Copying blob 35494cb8a948 skipped: already exists  
Copying blob 9c1b809bcfef skipped: already exists  
Copying blob 2794323d954c skipped: already exists  
WARN[0139] Failed, retrying in 1s ... (2/3). Error: writing blob: Patch "https://quay3121hitachi-quay-quay3121hcpv97.apps.quaytest-2954.qe.devcluster.openshift.com/v2/quay/python3/blobs/uploads/602736f6-c1af-4bfa-ae26-1477e75264b1": EOF 
Getting image source signatures
Copying blob 35494cb8a948 skipped: already exists  
Copying blob 2794323d954c skipped: already exists  
Copying blob 076d341e6674 skipped: already exists  
Copying blob 4af5b82c5321 skipped: already exists  
Copying blob 28e03088bc15 done   | 
Copying blob 9c1b809bcfef skipped: already exists  
Copying blob baeb38530ae9 skipped: already exists  
Copying blob 4d9811414818 skipped: already exists  
WARN[0209] Failed, retrying in 1s ... (3/3). Error: writing blob: Patch "https://quay3121hitachi-quay-quay3121hcpv97.apps.quaytest-2954.qe.devcluster.openshift.com/v2/quay/python3/blobs/uploads/7b9d695f-03f1-4e3d-bac5-0e0730ddf685": EOF 
Getting image source signatures
Copying blob 35494cb8a948 skipped: already exists  
Copying blob 076d341e6674 skipped: already exists  
Copying blob 28e03088bc15 done   | 
Copying blob 4af5b82c5321 skipped: already exists  
Copying blob 9c1b809bcfef skipped: already exists  
Copying blob 4d9811414818 skipped: already exists  
Copying blob 2794323d954c skipped: already exists  
Copying blob baeb38530ae9 skipped: already exists  
Error: writing blob: Patch "https://quay3121hitachi-quay-quay3121hcpv97.apps.quaytest-2954.qe.devcluster.openshift.com/v2/quay/python3/blobs/uploads/6ab17057-e757-4d9a-a080-44f8f14f5b6f": EOF


podman tag docker.io/library/postgres quay3121hitachi-quay-quay3121hcpv97.apps.quaytest-2954.qe.devcluster.openshift.com/quay/test123
podman push quay3121hitachi-quay-quay3121hcpv97.apps.quaytest-2954.qe.devcluster.openshift.com/quay/test123 --tls-verify=false --creds quay:***
Getting image source signatures
Copying blob 6aa2d5918ee5 done   | 
Copying blob 408c43c31618 done   | 
Copying blob 9853575bc4f9 done   | 
Copying blob 1841910efddb done   | 
Copying blob 9f42b3e949db done   | 
Copying blob 89b32b0e908d done   | 
Copying blob 9c9c71e4d581 done   | 
Copying blob 6a2213c8e247 done   | 
Copying blob d4c86fce3d7b done   | 
Copying blob c2e4b43b7613 done   | 
Copying blob 444f5e6f22aa done   | 
Copying blob 2ceca10bdc04 done   | 
Copying blob 1c0d86be3f17 done   | 
Copying blob 925ab77fab25 done   | 
WARN[0069] Failed, retrying in 1s ... (1/3). Error: writing blob: Patch "https://quay3121hitachi-quay-quay3121hcpv97.apps.quaytest-2954.qe.devcluster.openshift.com/v2/quay/test123/blobs/uploads/9cf5f0b4-e123-4b04-a9b3-42b7e63c083a": EOF 
Getting image source signatures
Copying blob 696d74f903c6 skipped: already exists  
Copying blob b24dee135565 skipped: already exists  
Copying blob c58f249d424f skipped: already exists  
Copying blob 12a637613cb0 skipped: already exists  
Copying blob 533072914e70 skipped: already exists  
Copying blob 85111e7c41c5 skipped: already exists  
Copying blob 3ec12691d09d skipped: already exists  
Copying blob d4c86fce3d7b done   | 
Copying blob 315909694129 skipped: already exists  
Copying blob 310a13d1e124 skipped: already exists  
Copying blob 3740eee731ae skipped: already exists  
Copying blob e1fbe10c5ebd skipped: already exists  
Copying blob 0169018f4f1d skipped: already exists  
Copying blob 97c4a2fe38a9 skipped: already exists  
WARN[0135] Failed, retrying in 1s ... (2/3). Error: writing blob: Patch "https://quay3121hitachi-quay-quay3121hcpv97.apps.quaytest-2954.qe.devcluster.openshift.com/v2/quay/test123/blobs/uploads/47a05c63-fd29-48db-892d-f3c0a7268ee4": EOF 
Getting image source signatures
Copying blob 85111e7c41c5 skipped: already exists  
Copying blob 12a637613cb0 skipped: already exists  
Copying blob 3ec12691d09d skipped: already exists  
Copying blob d4c86fce3d7b done   | 
Copying blob b24dee135565 skipped: already exists  
Copying blob 533072914e70 skipped: already exists  
Copying blob c58f249d424f skipped: already exists  
Copying blob 3740eee731ae skipped: already exists  
Copying blob e1fbe10c5ebd skipped: already exists  
Copying blob 310a13d1e124 skipped: already exists  
Copying blob 696d74f903c6 skipped: already exists  
Copying blob 0169018f4f1d skipped: already exists  
Copying blob 315909694129 skipped: already exists  
Copying blob 97c4a2fe38a9 skipped: already exists  
WARN[0201] Failed, retrying in 1s ... (3/3). Error: writing blob: Patch "https://quay3121hitachi-quay-quay3121hcpv97.apps.quaytest-2954.qe.devcluster.openshift.com/v2/quay/test123/blobs/uploads/ac004b5f-e6b2-4556-b369-1cb275b4f3a7": EOF 
Getting image source signatures
Copying blob 315909694129 skipped: already exists  
Copying blob 12a637613cb0 skipped: already exists  
Copying blob 3ec12691d09d skipped: already exists  
Copying blob 696d74f903c6 skipped: already exists  
Copying blob d4c86fce3d7b done   | 
Copying blob c58f249d424f skipped: already exists  
Copying blob b24dee135565 skipped: already exists  
Copying blob 3740eee731ae skipped: already exists  
Copying blob e1fbe10c5ebd skipped: already exists  
Copying blob 533072914e70 skipped: already exists  
Copying blob 0169018f4f1d skipped: already exists  
Copying blob 85111e7c41c5 skipped: already exists  
Copying blob 310a13d1e124 skipped: already exists  
Copying blob 97c4a2fe38a9 skipped: already exists  
Error: writing blob: Patch "https://quay3121hitachi-quay-quay3121hcpv97.apps.quaytest-2954.qe.devcluster.openshift.com/v2/quay/test123/blobs/uploads/45079214-a48f-4d3c-8f71-334023acba66": EOF

      Quay config.yaml:

      DEFAULT_TAG_EXPIRATION: 2w
DISTRIBUTED_STORAGE_CONFIG:
  local_us:
  - RadosGWStorage
  - access_key: ******
    bucket_name: redhat
    hostname: partner.hcpdemo.hitachivantara.com
    is_secure: true
    secret_key: ******
    storage_path: /datastorage/registry
    signature_version: v4
DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS:
- local_us
DISTRIBUTED_STORAGE_PREFERENCE:
- local_us
PERMANENTLY_DELETE_TAGS: true
RESET_CHILD_MANIFEST_EXPIRATION: true
FEATURE_EXTENDED_REPOSITORY_NAMES: true
CREATE_NAMESPACE_ON_PUSH: true
FEATURE_QUOTA_MANAGEMENT: true
FEATURE_PROXY_CACHE: true
FEATURE_USER_INITIALIZE: true
FEATURE_PROXY_STORAGE: true
FEATURE_UI_V2: true
FEATURE_SUPERUSERS_FULL_ACCESS: true
FEATURE_AUTO_PRUNE: true

              lzha1981 luffy zhang
              lzha1981 luffy zhang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: