Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-3916

Unable to complete mirroring sync (The creds are intermittently detected by the repomirror worker)

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Normal Normal
    • None
    • quay-v3.6.6
    • -area/repomirror, quay
    • False
    • None
    • False
    • Hide

      Password:
      Login Succeeded!
      sh-4.4#
      sh-4.4# skopeo --debug copy docker://registry.redhat.io/rhel8/buildah:latest docker://quay.lab.apps.cloud.internal/testkr/registry.redhat.io/rhel8/buildah:latest

      DEBU[0000] Loading registries configuration "/etc/containers/registries.conf.d/000-shortnames.conf"
      DEBU[0000] Loading registries configuration "/etc/containers/registries.conf.d/001-rhel-shortnames.conf"
      DEBU[0000] Loading registries configuration "/etc/containers/registries.conf.d/002-rhel-shortnames-overrides.conf"
      DEBU[0000] Found credentials for quay.lab.apps.cloud.internal in credential helper containers-auth.json
      DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration
      DEBU[0000] Using "default-docker" configuration
      DEBU[0000] Using file:///var/lib/containers/sigstore
      DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/quay.lab.apps.cloud.internal
      DEBU[0000] Trying to access "registry.redhat.io/rhel8/buildah:latest"
      DEBU[0000] Trying to access "registry.redhat.io/rhel8/buildah:latest"
      DEBU[0000] Found credentials for registry.redhat.io in credential helper containers-auth.json
      DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration
      DEBU[0000] Using "docker" namespace registry.redhat.io
      DEBU[0000] Using https://registry.redhat.io/containers/sigstore
      DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/registry.redhat.io
      DEBU[0000] GET https://registry.redhat.io/v2/
      DEBU[0000] Ping https://registry.redhat.io/v2/ status 401
      DEBU[0000] GET https://registry.redhat.io/auth/realms/rhcc/protocol/redhat-docker-v2/auth?account=880108%7Cquay-mirror&scope=repository%3Arhel8%2Fbuildah%3Apull&service=docker-registry
      DEBU[0000] GET https://registry.redhat.io/v2/rhel8/buildah/manifests/latest
      DEBU[0000] Content-Type from manifest GET is "application/vnd.docker.distribution.manifest.list.v2+json"
      DEBU[0000] Using blob info cache at /var/lib/containers/cache/blob-info-cache-v1.boltdb
      DEBU[0000] Source is a manifest list; copying (only) instance sha256:90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9 for current system
      DEBU[0000] GET https://registry.redhat.io/v2/rhel8/buildah/manifests/sha256:90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9
      DEBU[0001] Content-Type from manifest GET is "application/vnd.docker.distribution.manifest.v2+json"
      DEBU[0001] IsRunningImageAllowed for image docker:registry.redhat.io/rhel8/buildah:latest
      DEBU[0001] Using default policy section
      DEBU[0001] Requirement 0: allowed
      DEBU[0001] Overall: allowed
      Getting image source signatures
      DEBU[0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-1
      DEBU[0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-2
      DEBU[0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-3
      DEBU[0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-4
      DEBU[0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-5
      DEBU[0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-6
      DEBU[0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-7
      Checking if image destination supports signatures
      DEBU[0002] GET https://quay.lab.apps.cloud.internal/v2/
      DEBU[0002] Ping https://quay.lab.apps.cloud.internal/v2/ status 401
      DEBU[0002] We can't modify the manifest, hoping for the best...
      DEBU[0002] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:545277d800059b32cf03377a9301094e9ac8aa4bb42d809766d7355ca9aa8652
      DEBU[0002] GET https://quay.lab.apps.cloud.internal/v2/auth?account=testkr%2Brobot&scope=repository%3Atestkr%2Fregistry.redhat.io%2Frhel8%2Fbuildah%3Apull%2Cpush&service=quay.lab.apps.cloud.internal
      DEBU[0002] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:4f054d1a23f9e28e937882aef817356396d449c8f4f5e29903bff905d766683a
      DEBU[0002] GET https://quay.lab.apps.cloud.internal/v2/auth?account=testkr%2Brobot&scope=repository%3Atestkr%2Fregistry.redhat.io%2Frhel8%2Fbuildah%3Apull%2Cpush&service=quay.lab.apps.cloud.internal
      DEBU[0002] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:f70d60810c69edad990aaf0977a87c6d2bcc9cd52904fa6825f08507a9b6e7bc
      DEBU[0002] GET https://quay.lab.apps.cloud.internal/v2/auth?account=testkr%2Brobot&scope=repository%3Atestkr%2Fregistry.redhat.io%2Frhel8%2Fbuildah%3Apull%2Cpush&service=quay.lab.apps.cloud.internal
      DEBU[0002] Increasing token expiration to: 60 seconds
      DEBU[0002] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:f70d60810c69edad990aaf0977a87c6d2bcc9cd52904fa6825f08507a9b6e7bc
      DEBU[0002] Increasing token expiration to: 60 seconds
      DEBU[0002] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:545277d800059b32cf03377a9301094e9ac8aa4bb42d809766d7355ca9aa8652
      DEBU[0002] Increasing token expiration to: 60 seconds
      DEBU[0002] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:4f054d1a23f9e28e937882aef817356396d449c8f4f5e29903bff905d766683a
      DEBU[0002] ... already exists
      DEBU[0002] ... already exists
      DEBU[0002] Skipping blob sha256:4f054d1a23f9e28e937882aef817356396d449c8f4f5e29903bff905d766683a (already present):
      Copying blob 4f054d1a23f9 skipped: already exists
      Copying blob 4f054d1a23f9 skipped: already exists
      Copying blob 545277d80005 skipped: already exists
      Copying blob 4f054d1a23f9 skipped: already exists
      Copying blob 545277d80005 skipped: already exists
      Copying blob f70d60810c69 [--------------------------------------] 0.0b / 0.0b
      DEBU[0002] Downloading /v2/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332
      DEBU[0002] GET https://registry.redhat.io/v2/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332
      DEBU[0002] No compression detected
      DEBU[0002] Using original blob without modification
      DEBU[0002] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332
      DEBU[0002] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332
      DEBU[0002] ... already exists
      Copying config 671902eec1 [======================================] 4.6KiB / 4.6KiB
      Writing manifest to image destination
      DEBU[0002] PUT https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/manifests/latest
      Storing signatures
      DEBU[0002] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-1
      DEBU[0002] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-2
      DEBU[0002] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-3
      DEBU[0002] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-4
      DEBU[0002] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-5
      DEBU[0002] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-6
      DEBU[0002] Deleting /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-7
      sh-4.4# cat /run/containers/0/auth.json
      {
      "auths": {
      "quay.lab.apps.cloud.internal":

      { "auth": "dGVzdGtyK3JvYm90O<abc>T01CU0NIRTI=" }

      ,
      "registry.redhat.io":

      { "auth": "ODgwMTA4fHF1YXktbWlyZJc<abc>sQS1FRUl1aHY1OW0wQXNqU3ZV" }

      }
      }sh-4.4# skopeo --debug copy docker://registry.redhat.io/rhel8/buildah:latest docker://quay.lab.apps.cloud.internal/testkr/registry.redhat.io/hel8/buildah:123
      DEBU[0000] Loading registries configuration "/etc/containers/registries.conf"
      DEBU[0000] Loading registries configuration "/etc/containers/registries.conf.d/000-shortnames.conf"
      DEBU[0000] Loading registries configuration "/etc/containers/registries.conf.d/001-rhel-shortnames.conf"
      DEBU[0000] Loading registries configuration "/etc/containers/registries.conf.d/002-rhel-shortnames-overrides.conf"
      DEBU[0000] Found credentials for quay.lab.apps.cloud.internal in credential helper containers-auth.json
      DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration
      DEBU[0000] Using "default-docker" configuration
      DEBU[0000] Using file:///var/lib/containers/sigstore
      DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/quay.lab.apps.cloud.internal
      DEBU[0000] Trying to access "registry.redhat.io/rhel8/buildah:latest"
      DEBU[0000] Trying to access "registry.redhat.io/rhel8/buildah:latest"
      DEBU[0000] Found credentials for registry.redhat.io in credential helper containers-auth.json
      DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration
      DEBU[0000] Using "docker" namespace registry.redhat.io
      DEBU[0000] Using https://registry.redhat.io/containers/sigstore
      DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/registry.redhat.io
      DEBU[0000] GET https://registry.redhat.io/v2/
      DEBU[0000] Ping https://registry.redhat.io/v2/ status 401
      DEBU[0000] GET https://registry.redhat.io/auth/realms/rhcc/protocol/redhat-docker-v2/auth?account=880108%7Cquay-mirror&scope=repository%3Arhel8%2Fbuildah%3Apull&service=docker-registry
      DEBU[0000] GET https://registry.redhat.io/v2/rhel8/buildah/manifests/latest
      DEBU[0002] Content-Type from manifest GET is "application/vnd.docker.distribution.manifest.list.v2+json"
      DEBU[0002] Using blob info cache at /var/lib/containers/cache/blob-info-cache-v1.boltdb
      DEBU[0002] Source is a manifest list; copying (only) instance sha256:90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9 for current system
      DEBU[0002] GET https://registry.redhat.io/v2/rhel8/buildah/manifests/sha256:90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9
      DEBU[0003] Content-Type from manifest GET is "application/vnd.docker.distribution.manifest.v2+json"
      DEBU[0003] IsRunningImageAllowed for image docker:registry.redhat.io/rhel8/buildah:latest
      DEBU[0003] Using default policy section
      DEBU[0003] Requirement 0: allowed
      DEBU[0003] Overall: allowed
      Getting image source signatures
      DEBU[0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-1
      DEBU[0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-2
      DEBU[0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-3
      DEBU[0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-4
      DEBU[0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-5
      DEBU[0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-6
      DEBU[0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-7
      Checking if image destination supports signatures
      DEBU[0004] GET https://quay.lab.apps.cloud.internal/v2/
      DEBU[0004] Ping https://quay.lab.apps.cloud.internal/v2/ status 401
      DEBU[0004] We can't modify the manifest, hoping for the best...
      DEBU[0004] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:545277d800059b32cf03377a9301094e9ac8aa4bb42d809766d7355ca9aa8652
      DEBU[0004] GET https://quay.lab.apps.cloud.internal/v2/auth?account=testkr%2Brobot&scope=repository%3Atestkr%2Fregistry.redhat.io%2Frhel8%2Fbuildah%3Apull%2Cpush&service=quay.lab.apps.cloud.internal
      DEBU[0004] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:4f054d1a23f9e28e937882aef817356396d449c8f4f5e29903bff905d766683a
      DEBU[0004] GET https://quay.lab.apps.cloud.internal/v2/auth?account=testkr%2Brobot&scope=repository%3Atestkr%2Fregistry.redhat.io%2Frhel8%2Fbuildah%3Apull%2Cpush&service=quay.lab.apps.cloud.internal
      DEBU[0004] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:f70d60810c69edad990aaf0977a87c6d2bcc9cd52904fa6825f08507a9b6e7bc
      DEBU[0004] GET https://quay.lab.apps.cloud.internal/v2/auth?account=testkr%2Brobot&scope=repository%3Atestkr%2Fregistry.redhat.io%2Frhel8%2Fbuildah%3Apull%2Cpush&service=quay.lab.apps.cloud.internal
      DEBU[0004] Increasing token expiration to: 60 seconds
      DEBU[0004] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:4f054d1a23f9e28e937882aef817356396d449c8f4f5e29903bff905d766683a
      DEBU[0004] Increasing token expiration to: 60 seconds
      DEBU[0004] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:545277d800059b32cf03377a9301094e9ac8aa4bb42d809766d7355ca9aa8652
      DEBU[0004] Increasing token expiration to: 60 seconds
      DEBU[0004] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:f70d60810c69edad990aaf0977a87c6d2bcc9cd52904fa6825f08507a9b6e7bc
      DEBU[0004] ... already exists
      DEBU[0004] Skipping blob sha256:4f054d1a23f9e28e937882aef817356396d449c8f4f5e29903bff905d766683a (already present):
      Copying blob 4f054d1a23f9 skipped: already exists
      DEBU[0004] ... already exists
      Copying blob 4f054d1a23f9 skipped: already exists
      Copying blob 545277d80005 skipped: already exists
      Copying blob 4f054d1a23f9 skipped: already exists
      Copying blob 545277d80005 skipped: already exists
      Copying blob f70d60810c69 [--------------------------------------] 0.0b / 0.0b
      DEBU[0004] Downloading /v2/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332
      DEBU[0004] GET https://registry.redhat.io/v2/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332
      DEBU[0004] No compression detected
      DEBU[0004] Using original blob without modification
      DEBU[0004] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332
      DEBU[0004] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332
      DEBU[0004] ... already exists
      Copying config 671902eec1 [======================================] 4.6KiB / 4.6KiB
      Writing manifest to image destination
      DEBU[0004] PUT https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/manifests/123
      Storing signatures
      DEBU[0004] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-1
      DEBU[0004] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-2
      DEBU[0004] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-3
      DEBU[0004] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-4
      DEBU[0004] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-5
      DEBU[0004] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-6
      DEBU[0004] Deleting /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-7
      sh-4.4#

      Show
      Password: Login Succeeded! sh-4.4# sh-4.4# skopeo --debug copy docker://registry.redhat.io/rhel8/buildah:latest docker://quay.lab.apps.cloud.internal/testkr/registry.redhat.io/rhel8/buildah:latest DEBU [0000] Loading registries configuration "/etc/containers/registries.conf.d/000-shortnames.conf" DEBU [0000] Loading registries configuration "/etc/containers/registries.conf.d/001-rhel-shortnames.conf" DEBU [0000] Loading registries configuration "/etc/containers/registries.conf.d/002-rhel-shortnames-overrides.conf" DEBU [0000] Found credentials for quay.lab.apps.cloud.internal in credential helper containers-auth.json DEBU [0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration DEBU [0000] Using "default-docker" configuration DEBU [0000] Using file:///var/lib/containers/sigstore DEBU [0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/quay.lab.apps.cloud.internal DEBU [0000] Trying to access "registry.redhat.io/rhel8/buildah:latest" DEBU [0000] Trying to access "registry.redhat.io/rhel8/buildah:latest" DEBU [0000] Found credentials for registry.redhat.io in credential helper containers-auth.json DEBU [0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration DEBU [0000] Using "docker" namespace registry.redhat.io DEBU [0000] Using https://registry.redhat.io/containers/sigstore DEBU [0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/registry.redhat.io DEBU [0000] GET https://registry.redhat.io/v2/ DEBU [0000] Ping https://registry.redhat.io/v2/ status 401 DEBU [0000] GET https://registry.redhat.io/auth/realms/rhcc/protocol/redhat-docker-v2/auth?account=880108%7Cquay-mirror&scope=repository%3Arhel8%2Fbuildah%3Apull&service=docker-registry DEBU [0000] GET https://registry.redhat.io/v2/rhel8/buildah/manifests/latest DEBU [0000] Content-Type from manifest GET is "application/vnd.docker.distribution.manifest.list.v2+json" DEBU [0000] Using blob info cache at /var/lib/containers/cache/blob-info-cache-v1.boltdb DEBU [0000] Source is a manifest list; copying (only) instance sha256:90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9 for current system DEBU [0000] GET https://registry.redhat.io/v2/rhel8/buildah/manifests/sha256:90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9 DEBU [0001] Content-Type from manifest GET is "application/vnd.docker.distribution.manifest.v2+json" DEBU [0001] IsRunningImageAllowed for image docker:registry.redhat.io/rhel8/buildah:latest DEBU [0001] Using default policy section DEBU [0001] Requirement 0: allowed DEBU [0001] Overall: allowed Getting image source signatures DEBU [0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-1 DEBU [0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-2 DEBU [0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-3 DEBU [0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-4 DEBU [0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-5 DEBU [0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-6 DEBU [0001] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-7 Checking if image destination supports signatures DEBU [0002] GET https://quay.lab.apps.cloud.internal/v2/ DEBU [0002] Ping https://quay.lab.apps.cloud.internal/v2/ status 401 DEBU [0002] We can't modify the manifest, hoping for the best... DEBU [0002] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:545277d800059b32cf03377a9301094e9ac8aa4bb42d809766d7355ca9aa8652 DEBU [0002] GET https://quay.lab.apps.cloud.internal/v2/auth?account=testkr%2Brobot&scope=repository%3Atestkr%2Fregistry.redhat.io%2Frhel8%2Fbuildah%3Apull%2Cpush&service=quay.lab.apps.cloud.internal DEBU [0002] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:4f054d1a23f9e28e937882aef817356396d449c8f4f5e29903bff905d766683a DEBU [0002] GET https://quay.lab.apps.cloud.internal/v2/auth?account=testkr%2Brobot&scope=repository%3Atestkr%2Fregistry.redhat.io%2Frhel8%2Fbuildah%3Apull%2Cpush&service=quay.lab.apps.cloud.internal DEBU [0002] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:f70d60810c69edad990aaf0977a87c6d2bcc9cd52904fa6825f08507a9b6e7bc DEBU [0002] GET https://quay.lab.apps.cloud.internal/v2/auth?account=testkr%2Brobot&scope=repository%3Atestkr%2Fregistry.redhat.io%2Frhel8%2Fbuildah%3Apull%2Cpush&service=quay.lab.apps.cloud.internal DEBU [0002] Increasing token expiration to: 60 seconds DEBU [0002] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:f70d60810c69edad990aaf0977a87c6d2bcc9cd52904fa6825f08507a9b6e7bc DEBU [0002] Increasing token expiration to: 60 seconds DEBU [0002] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:545277d800059b32cf03377a9301094e9ac8aa4bb42d809766d7355ca9aa8652 DEBU [0002] Increasing token expiration to: 60 seconds DEBU [0002] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:4f054d1a23f9e28e937882aef817356396d449c8f4f5e29903bff905d766683a DEBU [0002] ... already exists DEBU [0002] ... already exists DEBU [0002] Skipping blob sha256:4f054d1a23f9e28e937882aef817356396d449c8f4f5e29903bff905d766683a (already present): Copying blob 4f054d1a23f9 skipped: already exists Copying blob 4f054d1a23f9 skipped: already exists Copying blob 545277d80005 skipped: already exists Copying blob 4f054d1a23f9 skipped: already exists Copying blob 545277d80005 skipped: already exists Copying blob f70d60810c69 [--------------------------------------] 0.0b / 0.0b DEBU [0002] Downloading /v2/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332 DEBU [0002] GET https://registry.redhat.io/v2/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332 DEBU [0002] No compression detected DEBU [0002] Using original blob without modification DEBU [0002] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332 DEBU [0002] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332 DEBU [0002] ... already exists Copying config 671902eec1 [======================================] 4.6KiB / 4.6KiB Writing manifest to image destination DEBU [0002] PUT https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/manifests/latest Storing signatures DEBU [0002] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-1 DEBU [0002] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-2 DEBU [0002] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-3 DEBU [0002] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-4 DEBU [0002] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-5 DEBU [0002] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-6 DEBU [0002] Deleting /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-7 sh-4.4# cat /run/containers/0/auth.json { "auths": { "quay.lab.apps.cloud.internal": { "auth": "dGVzdGtyK3JvYm90O<abc>T01CU0NIRTI=" } , "registry.redhat.io": { "auth": "ODgwMTA4fHF1YXktbWlyZJc<abc>sQS1FRUl1aHY1OW0wQXNqU3ZV" } } }sh-4.4# skopeo --debug copy docker://registry.redhat.io/rhel8/buildah:latest docker://quay.lab.apps.cloud.internal/testkr/registry.redhat.io/hel8/buildah:123 DEBU [0000] Loading registries configuration "/etc/containers/registries.conf" DEBU [0000] Loading registries configuration "/etc/containers/registries.conf.d/000-shortnames.conf" DEBU [0000] Loading registries configuration "/etc/containers/registries.conf.d/001-rhel-shortnames.conf" DEBU [0000] Loading registries configuration "/etc/containers/registries.conf.d/002-rhel-shortnames-overrides.conf" DEBU [0000] Found credentials for quay.lab.apps.cloud.internal in credential helper containers-auth.json DEBU [0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration DEBU [0000] Using "default-docker" configuration DEBU [0000] Using file:///var/lib/containers/sigstore DEBU [0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/quay.lab.apps.cloud.internal DEBU [0000] Trying to access "registry.redhat.io/rhel8/buildah:latest" DEBU [0000] Trying to access "registry.redhat.io/rhel8/buildah:latest" DEBU [0000] Found credentials for registry.redhat.io in credential helper containers-auth.json DEBU [0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration DEBU [0000] Using "docker" namespace registry.redhat.io DEBU [0000] Using https://registry.redhat.io/containers/sigstore DEBU [0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/registry.redhat.io DEBU [0000] GET https://registry.redhat.io/v2/ DEBU [0000] Ping https://registry.redhat.io/v2/ status 401 DEBU [0000] GET https://registry.redhat.io/auth/realms/rhcc/protocol/redhat-docker-v2/auth?account=880108%7Cquay-mirror&scope=repository%3Arhel8%2Fbuildah%3Apull&service=docker-registry DEBU [0000] GET https://registry.redhat.io/v2/rhel8/buildah/manifests/latest DEBU [0002] Content-Type from manifest GET is "application/vnd.docker.distribution.manifest.list.v2+json" DEBU [0002] Using blob info cache at /var/lib/containers/cache/blob-info-cache-v1.boltdb DEBU [0002] Source is a manifest list; copying (only) instance sha256:90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9 for current system DEBU [0002] GET https://registry.redhat.io/v2/rhel8/buildah/manifests/sha256:90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9 DEBU [0003] Content-Type from manifest GET is "application/vnd.docker.distribution.manifest.v2+json" DEBU [0003] IsRunningImageAllowed for image docker:registry.redhat.io/rhel8/buildah:latest DEBU [0003] Using default policy section DEBU [0003] Requirement 0: allowed DEBU [0003] Overall: allowed Getting image source signatures DEBU [0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-1 DEBU [0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-2 DEBU [0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-3 DEBU [0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-4 DEBU [0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-5 DEBU [0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-6 DEBU [0003] GET https://registry.redhat.io/containers/sigstore/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-7 Checking if image destination supports signatures DEBU [0004] GET https://quay.lab.apps.cloud.internal/v2/ DEBU [0004] Ping https://quay.lab.apps.cloud.internal/v2/ status 401 DEBU [0004] We can't modify the manifest, hoping for the best... DEBU [0004] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:545277d800059b32cf03377a9301094e9ac8aa4bb42d809766d7355ca9aa8652 DEBU [0004] GET https://quay.lab.apps.cloud.internal/v2/auth?account=testkr%2Brobot&scope=repository%3Atestkr%2Fregistry.redhat.io%2Frhel8%2Fbuildah%3Apull%2Cpush&service=quay.lab.apps.cloud.internal DEBU [0004] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:4f054d1a23f9e28e937882aef817356396d449c8f4f5e29903bff905d766683a DEBU [0004] GET https://quay.lab.apps.cloud.internal/v2/auth?account=testkr%2Brobot&scope=repository%3Atestkr%2Fregistry.redhat.io%2Frhel8%2Fbuildah%3Apull%2Cpush&service=quay.lab.apps.cloud.internal DEBU [0004] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:f70d60810c69edad990aaf0977a87c6d2bcc9cd52904fa6825f08507a9b6e7bc DEBU [0004] GET https://quay.lab.apps.cloud.internal/v2/auth?account=testkr%2Brobot&scope=repository%3Atestkr%2Fregistry.redhat.io%2Frhel8%2Fbuildah%3Apull%2Cpush&service=quay.lab.apps.cloud.internal DEBU [0004] Increasing token expiration to: 60 seconds DEBU [0004] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:4f054d1a23f9e28e937882aef817356396d449c8f4f5e29903bff905d766683a DEBU [0004] Increasing token expiration to: 60 seconds DEBU [0004] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:545277d800059b32cf03377a9301094e9ac8aa4bb42d809766d7355ca9aa8652 DEBU [0004] Increasing token expiration to: 60 seconds DEBU [0004] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:f70d60810c69edad990aaf0977a87c6d2bcc9cd52904fa6825f08507a9b6e7bc DEBU [0004] ... already exists DEBU [0004] Skipping blob sha256:4f054d1a23f9e28e937882aef817356396d449c8f4f5e29903bff905d766683a (already present): Copying blob 4f054d1a23f9 skipped: already exists DEBU [0004] ... already exists Copying blob 4f054d1a23f9 skipped: already exists Copying blob 545277d80005 skipped: already exists Copying blob 4f054d1a23f9 skipped: already exists Copying blob 545277d80005 skipped: already exists Copying blob f70d60810c69 [--------------------------------------] 0.0b / 0.0b DEBU [0004] Downloading /v2/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332 DEBU [0004] GET https://registry.redhat.io/v2/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332 DEBU [0004] No compression detected DEBU [0004] Using original blob without modification DEBU [0004] Checking /v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332 DEBU [0004] HEAD https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/blobs/sha256:671902eec1c557a06dacc7750b3991443993e77449f7aa4a5789b7a781fdf332 DEBU [0004] ... already exists Copying config 671902eec1 [======================================] 4.6KiB / 4.6KiB Writing manifest to image destination DEBU [0004] PUT https://quay.lab.apps.cloud.internal/v2/testkr/registry.redhat.io/rhel8/buildah/manifests/123 Storing signatures DEBU [0004] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-1 DEBU [0004] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-2 DEBU [0004] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-3 DEBU [0004] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-4 DEBU [0004] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-5 DEBU [0004] Writing to /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-6 DEBU [0004] Deleting /var/lib/containers/sigstore/testkr/registry.redhat.io/rhel8/buildah@sha256=90d5dbf6ceb3524e8bf6a401bca7ba04b608c6494c589902bc22aa10662347f9/signature-7 sh-4.4#
    • Customer Facing

      we have Quay 3.6.6 running on a OpenShift Cluster 4.7. When I try to sync the repo

      registry.redhat.io/rhel8/buildah

      for the tags *, then all the images are downloaded and just before finishing the sync all images except the latest are deleted, at the end I only see the latest. Then I sync the tags *,8.6 and all images of the repo were successfully synced.

       

       

        1. quay-quay-mirror-6458cdb9cc-57rcp_20220603.txt
          41.21 MB
          Carlos Eduardo da Silva
        2. quay-quay-mirror-6458cdb9cc-57rcp.log
          9.89 MB
          Carlos Eduardo da Silva
        3. quay-quay-mirror-6458cdb9cc-s5wrx_20220603.txt
          694 kB
          Carlos Eduardo da Silva
        4. quay-quay-mirror-6458cdb9cc-s5wrx.log
          17.41 MB
          Carlos Eduardo da Silva

              Unassigned Unassigned
              cdasilva@redhat.com Carlos Eduardo da Silva
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: