Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-78145

HCP Control Plane Failure due to Missing SCC Pinning and PSA Conflict

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • 4.21
    • HyperShift
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      In an HCP architecture, the control plane pods run within a namespace on a Management Cluster. This namespace is typically labeled with pod-security.kubernetes.io/enforce: restricted.
      
      If an administrative action on the Management Cluster grants a permissive SCC (such as anyuid) to the system:authenticated group, the SCC admission controller will prioritize this permissive SCC for the unpinned HCP pods. However, because the namespace is under a restricted PSA profile, the resulting pod specifications violate security standards (e.g., allowPrivilegeEscalation != false, runAsNonRoot != true).
      
      Consequently, the ReplicaSet controller fails to create pods, resulting in a permanent FailedCreate state for the hosted control plane.
      
      

      Version-Release number of selected component (if applicable):

          All HCP versions

      How reproducible:

          100%

      Steps to Reproduce:

         1. Deploy an HCP cluster
      
         2. Apply a global RBAC change on the Management Cluster
      
            oc adm policy add-scc-to-group anyuid system:authenticated
       
         3. Trigger a Control Plane Rollout
      
            oc delete pods --all -n <hcp-namespace>
      
         4. Observe the Failure
      
            # Check deployment status - most will show 0/1 READY
            oc get deploy -n <hcp-namespace>
      
            # Inspect the events for a failing ReplicaSet (e.g., kube-apiserver or etcd)
            oc get events -n <hcp-namespace> --sort-by='.lastTimestamp' 
       
         5. Verify the Pinned Pods
      
            Observe that pods with the openshift.io/required-scc annotation (like the aws-ebs-csi-driver) successfully bypassed this conflict and are running.
            oc get deploy aws-ebs-csi-driver-controller -n <hcp-namespace>
      
           

      Actual results:

      You will see a FailedCreate warning. 
      
      The SCC Admission controller selected anyuid, but the Pod Security Admission (PSA) controller (set to restricted by default on HCP namespaces) blocked it:
      
      Error creating: pods "..." is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true...
      
      

      Expected results:

      The HCP control plane pods must have the `openshift.io/required-scc` annotation to pin the required SCC (e.g., restricted-v2).
      
      The admission controller will ignore permissive global SCCs, ensuring the pods always meet the Namespace PSA requirements regardless of external configuration drift.
      
      

      Additional info:

         Workaround to fix this is removing the anyuid scc from the system:authenticated group
      
      
      oc adm policy remove-scc-from-group anyuid system:authenticated
      

       

              Unassigned Unassigned
              rhn-support-dpateriy Divyam Pateriya
              Yu Li Yu Li
              None
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: