Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-3901

Clair 4.4.2 failed to fetch image layer from quay when image was from dockerhub

XMLWordPrintable

    • False
    • None
    • False
    • 0

      Description:

      This is an issue found after push some images to Quay 3.7.1,  the results is can't see the image vulnerability scan results, checked Clair APP POD, get error message ""{"level":"warn","component":"internal/indexer/controller/Controller.Index","manifest":"sha256:bfc3ba0cc5604df7e13d85e39a2a27a1dc93885f24184423524adf50a4ecd916","state":"FetchLayers","error":"encountered error while fetching a layer: fetcher: unexpected status code: 500 Internal Server Error (body starts: \"<html> r n<head><title>500 Internal Server Error</title></head> r n<body bgcolor=\\\"white\\\"> r n<center><h1>500 Internal Server Error</h1></center>\\r\\n<hr><center>nginx/1.14.1</center>\\r\\n</body>\\r\\n</html>\\r\\n\")","time":"2022-06-02T03:27:51Z","message":"layers fetch failure"}, see attached Clair APP POD logs quay371_clair_app.logs 

      Quay Image: quay-operator-bundle-container-v3.7.1-9

      Quay 3.7.1 Console:

      {"level":"info","component":"httptransport/New","remote_addr":"10.129.3.227:55260","method":"POST","request_uri":"/indexer/api/v1/index_report","status":500,"duration":249.696853,"time":"2022-06-02T03:27:53Z","message":"handled HTTP request"}
      {"level":"info","manifest":"sha256:cfdb19c8fdc4b5ff3f5ed39b6958b1ac6aab0d41a39b00b15eafa9f3dab4e405","component":"libindex/Libindex.Index","time":"2022-06-02T03:27:53Z","message":"index request start"}
      {"level":"info","component":"internal/indexer/controller/Controller.Index","manifest":"sha256:cfdb19c8fdc4b5ff3f5ed39b6958b1ac6aab0d41a39b00b15eafa9f3dab4e405","time":"2022-06-02T03:27:53Z","message":"starting scan"}
      {"level":"info","component":"internal/indexer/controller/Controller.Index","manifest":"sha256:cfdb19c8fdc4b5ff3f5ed39b6958b1ac6aab0d41a39b00b15eafa9f3dab4e405","state":"CheckManifest","time":"2022-06-02T03:27:53Z","message":"manifest to be scanned"}
      {"level":"info","component":"internal/indexer/controller/Controller.Index","manifest":"sha256:cfdb19c8fdc4b5ff3f5ed39b6958b1ac6aab0d41a39b00b15eafa9f3dab4e405","state":"FetchLayers","time":"2022-06-02T03:27:53Z","message":"layers fetch start"}
      {"level":"warn","state":"FetchLayers","component":"internal/indexer/controller/Controller.Index","manifest":"sha256:cfdb19c8fdc4b5ff3f5ed39b6958b1ac6aab0d41a39b00b15eafa9f3dab4e405","error":"encountered error while fetching a layer: fetcher: unexpected status code: 500 Internal Server Error (body starts: \"<html>\\r\\n<head><title>500 Internal Server Error</title></head>\\r\\n<body bgcolor=\\\"white\\\">\\r\\n<center><h1>500 Internal Server Error</h1></center>\\r\\n<hr><center>nginx/1.14.1</center>\\r\\n</body>\\r\\n</html>\\r\\n\")","time":"2022-06-02T03:27:53Z","message":"layers fetch failure"}

      Steps:

      1. Deploy Quay 3.7.1 with Managed Clair Component, use AWS S3 as backend storage
      2. Push images from "docker.io/postgres" with latest tag to Quay new image repo
      3. Check Image vulnerability scan results

      Expected Results:

      Can see the Image vulnerability scan results.

      Actual Results:

      Can't see the Image vulnerability scan results.

      Additional info:

      When image was from registry.redhat.io, clair can fetch layer file and send vulnerabilities to quay

        1. image-2022-06-02-11-19-48-363.png
          image-2022-06-02-11-19-48-363.png
          209 kB
        2. image-2022-06-02-11-49-38-714.png
          image-2022-06-02-11-49-38-714.png
          226 kB
        3. image-2022-06-06-12-13-18-187.png
          image-2022-06-06-12-13-18-187.png
          231 kB
        4. image-2022-06-06-12-13-46-675.png
          image-2022-06-06-12-13-46-675.png
          232 kB
        5. image-2022-06-06-12-14-11-522.png
          image-2022-06-06-12-14-11-522.png
          317 kB
        6. image-2022-06-07-11-07-00-433.png
          image-2022-06-07-11-07-00-433.png
          395 kB
        7. image-2022-06-10-13-28-22-176.png
          image-2022-06-10-13-28-22-176.png
          415 kB
        8. quay371_app_pod1.logs
          15.32 MB
        9. quay371_app_pod2.logs
          4.24 MB
        10. quay371_clair_app.logs
          921 kB

            doconnor@redhat.com Dave O'Connor
            lzha1981 luffy zhang
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: