Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-6603

Quay Container Security Operator: Thousands of etcd objects each 1mb + in size filling etcd db: imagemanifestvulns

XMLWordPrintable

    • False
    • None
    • False
    • User Experience
    • Critical

      Issue:
      OpenShift etcd database is constantly exceeding threshold in size for defragmentation
      Thousands of objects are created of type:
      /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image-name>/sha256.<sha-string>

      Objectively need to understand if there is a way to reduce object size or count, as these objects are .5mb --> 1.5MB in size and there are thousands of them which is putting pressure on the database.

      See sample output:

      ~~~
      sort -k2,2nr 03694241_etcdkeys.txt | head -n 20

      514103841 1572183 327 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      91176116 1572002 58 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      113183712 1571996 72 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      9430968 1571828 6 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      370945036 1571801 236 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      301653120 1571110 192 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      9422082 1570347 6 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      3138924 1569462 2 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      208379745 1566765 133 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      203260460 1563542 130 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      9375888 1562648 6 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      129467633 1559851 83 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      118548068 1559843 76 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      389766000 1559064 250 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      213328728 1557144 137 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>sha256.<string>
      194502250 1556018 125 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      198978432 1554519 128 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      166318660 1554380 107 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      209179665 1549479 135 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      252560187 1549449 163 /kubernetes.io/secscan.quay.redhat.com/imagemanifestvulns/<image>/sha256.<string>
      ~~~

      Second column here is size in bytes for each file. There are thousands of entries - need to be sure this is expected and ask about how to reduce size/item count. Plausibly reduction of image count will be the way forward but need engineering confirmation that this is expected with high image count. I am not sure this is the correct jira group/target but can't find a better - please let me know if I should file this elsewhere.

              Unassigned Unassigned
              rhn-support-wrussell Will Russell
              Jonathan King
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: