Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-3037

[apiserver-auth] default SCC restricted allow volumes don't have "ephemeral" caused deployment with Generic Ephemeral Volumes stuck at Pending

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Undefined
    • 4.12.z
    • 4.12.0
    • Storage / Kubernetes
    • None

    Description

      This bug is a backport clone of [Bugzilla Bug 2100429](https://bugzilla.redhat.com/show_bug.cgi?id=2100429). The following is the description of the original bug:

      Description of problem:
      [apiserver-auth] default SCC restricted allow volumes don't have "ephemeral" caused deployment with Generic Ephemeral Volumes stuck at Pending

      Version-Release number of selected component (if applicable):
      Cluster version is 4.11.0-0.nightly-2022-06-22-190830
      $ oc version
      Client Version: 4.11.0-0.nightly-2022-05-11-054135
      Kustomize Version: v4.5.4
      Server Version: 4.11.0-0.nightly-2022-06-22-190830
      Kubernetes Version: v1.24.0+284d62a

      How reproducible:
      Always

      Steps to Reproduce:

      1. Set up a AWS OCP cluster with 4.11 nightly
      2. Create a deployment with Generic Ephemeral Volumes
      3. Waiting for the deployment ready and check the volume could write and read data

      Test data:
      wangpenghao@MacBook-Pro ~ cat temp.yaml
      apiVersion: apps/v1
      kind: Deployment
      metadata:
      name: my-dep
      spec:
      replicas: 1
      selector:
      matchLabels:
      app: my-dep
      template:
      metadata:
      labels:
      app: my-dep
      spec:
      containers:

      • image: >-
        quay.io/openshifttest/hello-openshift@sha256:b1aabe8c8272f750ce757b6c4263a2712796297511e0c6df79144ee188933623
        name: my-container
        ports:
      • containerPort: 80
        volumeMounts:
      • mountPath: /mnt/storage
        name: inline-volume
        volumes:
      • name: inline-volume
        ephemeral:
        volumeClaimTemplate:
        metadata:
        labels:
        workloadName: my-dep
        spec:
        accessModes:
      • ReadWriteOnce
        storageClassName: gp3-csi
        resources:
        requests:
        storage: 1Gi
        wangpenghao@MacBook-Pro ~ oc apply -f temp.yaml
        Warning: would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "my-dep-mcxx803w" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "my-dep-mcxx803w" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "my-dep-mcxx803w" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "my-dep-mcxx803w" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
        deployment.apps/my-dep created
        wangpenghao@MacBook-Pro ~ oc get deploy
        NAME READY UP-TO-DATE AVAILABLE AGE
        my-dep 0/1 0 0 7s
        wangpenghao@MacBook-Pro ~ oc get event
        LAST SEEN TYPE REASON OBJECT MESSAGE
        5s Warning FailedCreate replicaset/my-dep-6bd958d877 Error creating: pods "my-dep-6bd958d877-" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount, spec.volumes[0]: Invalid value: "ephemeral": ephemeral volumes are not allowed to be used, provider "restricted": Forbidden: not usable by user or serviceaccount, provider "nonroot-v2": Forbidden: not usable by user or serviceaccount, provider "nonroot": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount, provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount, provider "hostnetwork-v2": Forbidden: not usable by user or serviceaccount, provider "hostnetwork": Forbidden: not usable by user or serviceaccount, provider "hostaccess": Forbidden: not usable by user or serviceaccount, provider "node-exporter": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount]
        16s Normal ScalingReplicaSet deployment/my-dep Scaled up replica set my-dep-6bd958d877 to 1

      Actual results:
      In Step3 : The deployment stucked at Pending caused by unable to validate against any security context constraint

      Expected results:
      In Step3 : The deployment should ready with the default scc restricted, the default scc restricted should allow
      volumes:

      • ephemeral

      Additional info:

      Generic ephemeral volumes are the safer option of these two - it just creates/deletes PVCs on behalf of users. And most users can already create PVCs.

      ephemeral type volume not in scc.volumes list definition
      https://docs.openshift.com/container-platform/4.10/authentication/managing-security-context-constraints.html#authorization-cont[…]ing-internal-oauth

      So currently if customers want to use ephemeral type volume have to use scc with:
      volumes:

      • '*'
        E.g. scc/privileged

      Discuss record: https://coreos.slack.com/archives/CB48XQ4KZ/p1655465586780419

      Generic Ephemeral Volumes docs:
      https://kubernetes.io/blog/2020/09/01/ephemeral-volumes-with-storage-capacity-tracking/#generic-ephemeral-volumes

      Master Log:

      Node Log (of failed PODs):

      PV Dump:

      PVC Dump:

      StorageClass Dump (if StorageClass used by PV/PVC):

      Attachments

        Activity

          People

            rhn-engineering-jsafrane Jan Safranek
            openshift-crt-jira-prow OpenShift Prow Bot
            Penghao Wang Penghao Wang
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: