Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-36376

Security Profiles Operator pods are frequently killed for OOM in big environments

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • 4.12, 4.16
    • None
    • Moderate
    • No
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      Customer is reporting that Security Profiles Operator three replicas are killed for OOM. The situation stabilize when the memory limit gets doubled. The problem seems to be that by default the operator monitors every namespace, as confirmed by the following log:

      2024-06-28T06:52:39.358127239Z I0628 06:52:39.358108       1 main.go:368] "watching all namespaces" logger="setup" 

       

      Considering the actual requests and limits of these pods:

                resources:
                  limits:
                    cpu: 500m
                    memory: 128Mi
                  requests:
                    cpu: 250m
                    memory: 50Mi 

      Therefore, in big environments where there are a lot of user namespaces and pods the security profiles operator pods require more memory to be able to work.

       

      Version-Release number of selected component (if applicable):

      latest operator version    

       

      How reproducible:

      Create a cluster, create about 90 or 100 namespace and deploy a sufficient quantity of simple pods in each namespace. After that you will notice that the memory usage of the Security Profiles Operator three replicas will increase a lot leading to instability. 

       

      Actual results:

      Security Profiles Operator three replicas are killed for OOM

       

      Expected results:

      Security Profiles Operator three replicas work fine

       

      Additional info:

      Probably it could be useful to let users configure which namespaces needs to be monitored 

              wenshen@redhat.com Vincent Shen
              rhn-support-gizzi Giovanni Luca Izzi
              Xiaojie Yuan Xiaojie Yuan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: