Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-4640

Quay timeout during /secscan/notification - Root Cause Fix

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Minor Minor
    • None
    • quay-v3.7.8
    • quay
    • False
    • None
    • False
    • Quay Enterprise
    • Known Issue
    • Hide

      Here are the steps followed by customer:

      1. I spawned a LAB with OCS on OpenTLC, then installed OCS Operator and Quay Operator.
      > Everything with default values.

      2. I created a user in Quay.

      3. I pulled 3 images from microsoft from the bastion :

      • podman pull mcr.microsoft.com/dotnet/aspnet:6.0
      • podman pull mcr.microsoft.com/dotnet/aspnet:6.0-alpine
      • podman pull mcr.microsoft.com/dotnet/aspnet@sha256:1f6a6f9e939fc75dccd50133e2ef8fb4a8ac1006d1b50b6e6efcde1e3b88a6fc

      4. I retagged and pushed them in the repo of the user I created.

      5. And after that it's pretty much sit back and relax.

      Show
      Here are the steps followed by customer: 1. I spawned a LAB with OCS on OpenTLC, then installed OCS Operator and Quay Operator. > Everything with default values. 2. I created a user in Quay. 3. I pulled 3 images from microsoft from the bastion : podman pull mcr.microsoft.com/dotnet/aspnet:6.0 podman pull mcr.microsoft.com/dotnet/aspnet:6.0-alpine podman pull mcr.microsoft.com/dotnet/aspnet@sha256:1f6a6f9e939fc75dccd50133e2ef8fb4a8ac1006d1b50b6e6efcde1e3b88a6fc 4. I retagged and pushed them in the repo of the user I created. 5. And after that it's pretty much sit back and relax.

      This is a follow up to https://issues.redhat.com/browse/PROJQUAY-4002, which implemented a fix for the issue by increasing timeouts. This issue is meant to fix the underlying issue which is suspected to be a time-of-check to time-of-use bug. See this slack conversation for more details: https://coreos.slack.com/archives/G7VFPAY7Q/p1666197968423889

      *Note* This should be tested with Quay v3.7.8, before the secscan timeout increase was released.

      --------------
      Clair notification is not working. Clair pod's log shows error:
      ~~~

      {"level":"error","deliverer":"webhook","component":"notifier/Delivery.Deliver","error":"failed to deliver notification: code: 400 status 400 Bad request","time":"2022-06-02T15:22:21Z","message":"encountered error on tick"}

      ~~~
      Quay pod's log shows error:
      ~~~
      nginx stdout | 10.81.10.2 () - - [02/Jun/2022:15:22:01 +0000] "POST /secscan/notification HTTP/1.1" 400 173 "-" "clair/v4" (0.001 465 -)
      ~~~

       

              Unassigned Unassigned
              doconnor@redhat.com Dave O'Connor
              Muhammad Selim Jahangir
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: