Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-14800

OpenShift Cluster Upgrade Fails w/ Red Hat Advanced Cluster Security for Kubernetes Operator

    XMLWordPrintable

Details

    • Moderate
    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None

    Description

      Description of problem:

      When attempting to perform an OpenShift cluster upgrade (Version 4.10.16 -> 4.10.58), the cluster upgrade never begins. A failure message is not returned to the console and it appears as if OpenShift ignores the upgrade request.
      
      What ends up happening is upon initiating the cluster upgrade, in the "openshift-cluster-version" namespace, the "version-xxx-xx" pod is unable to successfully complete and repeatedly fails. This pod appears to be dynamically created when a cluster upgrade is requested and requires write access to the root filesystem. 
      Looking at the version pod's YAML, it has an annotation referencing a Security Context Constraint called "stackrox-collector", which is owned by the Red Hat Advanced Cluster Security Operator (RHACS) currently installed on my cluster:
      
               openshift.io/scc: stackrox-collector
      
      The stackrox-collector SCC has a flag with readOnlyRootFilesystem set to true, after manually changing this SCC to set readOnlyRootFilesystem to false, the OCP upgrade starts and completes successfully. 
      
      I'm not sure why or how ACS is associating the stackrox-collector SCC with the "version-xxx-xx" pod but it looks to be preventing any OCP upgrades from occurring unless the readOnlyRootFilesystem flag in the SCC is manually set to false.

      Version-Release number of selected component (if applicable):

      4.10.16

      How reproducible:

      Consistently reproducible.

      Steps to Reproduce:

      1. Install the RedHat ACS operator
      2. Attempt to perform an OCP upgrade
      3. Upgrade is not performed and version pod fails in the openshift-cluster-version namespace 

      Actual results:

      OCP Upgrade never begins 

      Expected results:

      OCP Upgrade is started

      Additional info:

      This behavior is extremely similar to this red hat issue: https://access.redhat.com/solutions/6969777
      
      However, in my case the SCC preventing the upgrade is one that was created by the ACS operator and not one that was created custom by an administrator. 

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              lozcoc Frank Mitarotonda
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: