Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-27062

The spod pods are restarting from time to time due to OOM after upgrade from 4.14 to 4.15

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • 4.15.0
    • None
    • Important
    • No
    • False
    • Hide

      None

      Show
      None

      Description of problem:

       

      The cluster was upgraded from 4.14>4.15(migration SDN > OVN first, and then ocp upgrade). The spod pods are restarting from time to time due to OOM. 
      $ oc get pod
      NAME                                                  READY   STATUS    RESTARTS        AGE
      security-profiles-operator-7994c6fcdc-9sszq           1/1     Running   0               10h
      security-profiles-operator-7994c6fcdc-vch2h           1/1     Running   0               10h
      security-profiles-operator-7994c6fcdc-ztsff           1/1     Running   0               9h
      security-profiles-operator-webhook-689c99796d-cbf6g   1/1     Running   0               10h
      security-profiles-operator-webhook-689c99796d-klv8j   1/1     Running   0               10h
      security-profiles-operator-webhook-689c99796d-wpwtb   1/1     Running   0               10h
      spod-8jcn9                                            3/3     Running   22 (13m ago)    35h
      spod-cd5st                                            3/3     Running   25 (88m ago)    35h
      spod-db6zc                                            3/3     Running   23 (95m ago)    35h
      spod-gz9fg                                            3/3     Running   23 (116m ago)   35h   
      

      Version-Release number of selected component (if applicable):

       Install a Vsphere IPI SDN cluster Install SPO, install 2 selinuxprofiles Migrate from SDN to OVN  Upgrade to version 4.15    

      How reproducible:

      Only once, haven't try to reproduce

      Steps to Reproduce:

      1. Install a Vsphere IPI SDN cluster
      2. Install SPO, install 2 selinuxprofiles
      3. Migrate from SDN to OVN 
      4. Upgrade to version 4.15

      Actual results:

      1. Install a Vsphere IPI SDN cluster
      2. Install SPO, install 2 selinuxprofiles
      3. Migrate from SDN to OVN 
      4. Upgrade to version 4.15

      Actual results:

       

      The spod pods are restarting from time to time. 
      $ oc get pod
      NAME                                                  READY   STATUS    RESTARTS        AGE
      security-profiles-operator-7994c6fcdc-9sszq           1/1     Running   0               10h
      security-profiles-operator-7994c6fcdc-vch2h           1/1     Running   0               10h
      security-profiles-operator-7994c6fcdc-ztsff           1/1     Running   0               9h
      security-profiles-operator-webhook-689c99796d-cbf6g   1/1     Running   0               10h
      security-profiles-operator-webhook-689c99796d-klv8j   1/1     Running   0               10h
      security-profiles-operator-webhook-689c99796d-wpwtb   1/1     Running   0               10h
      spod-8jcn9                                            3/3     Running   22 (13m ago)    35h
      spod-cd5st                                            3/3     Running   25 (88m ago)    35h
      spod-db6zc                                            3/3     Running   23 (95m ago)    35h
      spod-gz9fg                                            3/3     Running   23 (116m ago)   35h
      Check the spods, and find below info:
         Last State:     Terminated
            Reason:       OOMKilled
            Exit Code:    137
            Started:      Thu, 11 Jan 2024 04:12:19 -0800
            Finished:     Thu, 11 Jan 2024 06:32:34 -0800
          Ready:          True
          Restart Count:  16    
      

      Expected results:

      The Security Profiles Operator should work well, no spod pods restarting from time to time.     

      Additional info:

          

            wenshen@redhat.com Vincent Shen
            xiyuan@redhat.com Xiaojie Yuan
            Xiaojie Yuan Xiaojie Yuan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: