Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-4200

Container Security Operator timing out when clair scanning takes long to complete

    XMLWordPrintable

Details

    • False
    • None
    • False
    • Release Notes, Interactive Demo/Tutorial, User Experience
    • Quay Enterprise
    • Hide

      To reproduce I have set up the following:

      • OCP 4.10 cluster with Red Hat Quay deployed through Quay Operator
      • Followed defaults in installation manual: https://access.redhat.com/documentation/en-us/red_hat_quay/3.7/html/deploy_red_hat_quay_on_openshift_with_the_quay_operator
      • Object storage: A standalone instance of the Multi-Cloud Object Gateway backed by a local Kubernetes PersistentVolume storage
      • Created a QuayRegistry and uploaded an image with high number of vulnerabilities: registry.redhat.io/rhel8/python-27:2.7-75.1584015436
      • Generation of Security Scan report in Quay UI takes some time and finds 207 High and 1022 fixable vulnerabilities
      • Made image public in QuayRegistry
      • Installed Container Security Operator and added Quay's self-signed certificate to the container-security-operator-extra-certs secret
      • Created a deployment/pod that uses the image with high number of vulnerabilities
      • OpenShift dashboard is not able to see vulnerabilities same as detected from Quay UI
      • if using another image with a reduced number of vulnerabilities (e.g. alpine) the CSO is showing correctly the detected vulnerabilities
      • suspect is that this issue depends on the generation of the report taking too long in Quay, hence the 504 Gateway Time-out error in CSO.
      Show
      To reproduce I have set up the following: OCP 4.10 cluster with Red Hat Quay deployed through Quay Operator Followed defaults in installation manual: https://access.redhat.com/documentation/en-us/red_hat_quay/3.7/html/deploy_red_hat_quay_on_openshift_with_the_quay_operator Object storage: A standalone instance of the Multi-Cloud Object Gateway backed by a local Kubernetes PersistentVolume storage Created a QuayRegistry and uploaded an image with high number of vulnerabilities: registry.redhat.io/rhel8/python-27:2.7-75.1584015436 Generation of Security Scan report in Quay UI takes some time and finds 207 High and 1022 fixable vulnerabilities Made image public in QuayRegistry Installed Container Security Operator and added Quay's self-signed certificate to the container-security-operator-extra-certs secret Created a deployment/pod that uses the image with high number of vulnerabilities OpenShift dashboard is not able to see vulnerabilities same as detected from Quay UI if using another image with a reduced number of vulnerabilities (e.g. alpine) the CSO is showing correctly the detected vulnerabilities suspect is that this issue depends on the generation of the report taking too long in Quay, hence the 504 Gateway Time-out error in CSO.
    • 0

    Description

      Container Security Operator timing out with errors like the following when image used by pod has too many security vulnerabilities:

      level=debug msg="Pod updated" key=test-quay/example-67c756dff4-7mcb4
      level=info msg="Garbage collecting unreferenced ImageManifestVulns" key=test-quay/example-67c756dff4-7mcb4
      level=error msg="Failed to sync layer data" key=test-quay/example-67c756dff4-7mcb4 err="Request returned non-200 response: 504 Gateway Time-out"

       

      To reproduce I have set up the following:

      • OCP 4.10 cluster with Red Hat Quay deployed through Quay Operator
      • Followed defaults in installation manual: https://access.redhat.com/documentation/en-us/red_hat_quay/3.7/html/deploy_red_hat_quay_on_openshift_with_the_quay_operator
      • Object storage: A standalone instance of the Multi-Cloud Object Gateway backed by a local Kubernetes PersistentVolume storage
      • Created a QuayRegistry and uploaded an image with high number of vulnerabilities: registry.redhat.io/rhel8/python-27:2.7-75.1584015436
      • Generation of Security Scan report in Quay UI takes some time and finds 207 High and 1022 fixable vulnerabilities
      • Made image public in QuayRegistry
      • Installed Container Security Operator and added Quay's self-signed certificate to the container-security-operator-extra-certs secret
      • Created a deployment/pod that uses the image with high number of vulnerabilities
      • OpenShift dashboard is not able to see vulnerabilities same as detected from Quay UI
      • if using another image with a reduced number of vulnerabilities (e.g. alpine) the CSO is showing correctly the detected vulnerabilities
      • suspect is that this issue depends on the generation of the report taking too long in Quay, hence the 504 Gateway Time-out error in CSO.

      Attachments

        Activity

          People

            Unassigned Unassigned
            fminafra-redhat Francesco Minafra
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: