Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-2929

Quay Mirror Job is lost when mirror images

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Major Major
    • None
    • quay-v3.6.2
    • -area/repomirror, quay
    • False
    • False

      Description:

      This is an issue found when use Quay 3.6.2 to mirror images from redhat official image registry, after enable the mirror Job, get error ""Mirror finished unsuccessfully for 'registry.redhat.io/rhscl/redis-5-rhel7' job lost", after checked the logs of Mirror POD, can't figure out why the mirror job is lost, here file this issue to request Quay DEV team to review the logs of mirror POD, to see what happened. 

      If this is env issue(example, quay mirror worker can't connect with remote source image registry), suggest print out more logs to help customers to identify the root cause easily.

      Quay 3.6.2 image:  quay-operator-bundle-container-v3.6.2-15

      Note:

      • This issue can't be reproduce every time
      oc get pod
      NAME                                          READY   STATUS      RESTARTS      AGE
      quay-operator.v3.6.2-549f99765d-s5c8r         1/1     Running     0             22m
      quay362-clair-app-58f8f87fd4-f95ls            1/1     Running     0             9m42s
      quay362-clair-app-58f8f87fd4-fwv85            1/1     Running     0             9m52s
      quay362-clair-app-58f8f87fd4-ls4s9            1/1     Running     0             3m50s
      quay362-clair-app-58f8f87fd4-rdcxf            1/1     Running     0             3m20s
      quay362-clair-postgres-77b7d5667f-t2g89       1/1     Running     1 (11m ago)   11m
      quay362-quay-app-784d87d758-l6j2h             1/1     Running     0             9m52s
      quay362-quay-app-784d87d758-qplxz             1/1     Running     0             9m52s
      quay362-quay-app-upgrade--1-cl28c             0/1     Completed   0             10m
      quay362-quay-config-editor-66c89d4dd6-f4dvg   1/1     Running     0             9m52s
      quay362-quay-database-56d74f6987-tcxsn        1/1     Running     0             11m
      quay362-quay-mirror-5d656d8f89-fn9p4          1/1     Running     0             9m44s
      quay362-quay-mirror-5d656d8f89-lggrq          1/1     Running     0             9m44s
      quay362-quay-postgres-init--1-ntjqk           0/1     Completed   0             9m55s
      quay362-quay-redis-85c4f6f486-prcqd           1/1     Running     0             11m 
      Quay 3.6.2 Mirror Job is lost:

      repomirrorworker stdout | 2021-12-10 06:14:40,808 [55] [WARNING] [apscheduler.scheduler] Execution of job "RepoMirrorWorker._process_mirrors (trigger: interval[0:00:30], next run at: 2021-12-10 06:14:40 UTC)" skipped: maximum number of running instances reached (1)
      repomirrorworker stdout | 2021-12-10 06:14:47,231 [55] [INFO] [workers.repomirrorworker] Source 'docker://registry.redhat.io/rhscl/redis-5-rhel7:5-16' successful sync.
      2021-12-10 06:14:50,236 INFO reaped unknown pid 1274
      2021-12-10 06:14:50,236 INFO reaped unknown pid 1276
      2021-12-10 06:14:50,236 INFO reaped unknown pid 1278
      2021-12-10 06:14:50,236 INFO reaped unknown pid 1280
      2021-12-10 06:14:50,236 INFO reaped unknown pid 1282
      2021-12-10 06:14:50,236 INFO reaped unknown pid 1284
      2021-12-10 06:14:50,237 INFO reaped unknown pid 1286
      2021-12-10 06:14:50,237 INFO reaped unknown pid 1288
      2021-12-10 06:14:50,237 INFO reaped unknown pid 1289
      2021-12-10 06:14:50,237 INFO reaped unknown pid 1292
      2021-12-10 06:14:50,237 INFO reaped unknown pid 1294
      2021-12-10 06:14:50,237 INFO reaped unknown pid 1296
      2021-12-10 06:14:50,237 INFO reaped unknown pid 1298
      2021-12-10 06:14:50,237 INFO reaped unknown pid 1299
      2021-12-10 06:14:50,237 INFO reaped unknown pid 1302 

        1. image-2021-12-10-15-01-10-199.png
          330 kB
          luffy zhang
        2. quay362_mirror_pod1.logs
          24 kB
          luffy zhang
        3. quay362_mirror_pod2.logs
          54 kB
          luffy zhang

              Unassigned Unassigned
              lzha1981 luffy zhang
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: