Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-9046

istio sidecar fails to start with securitycontextconstraints on

XMLWordPrintable

    • Moderate
    • None
    • Unspecified
    • ---

      Created attachment 1845105 [details]
      SMCP describe

      Created attachment 1845105 [details]
      SMCP describe

      Description of problem: istio sidecar will fail to start if securitycontextconstraints cluster role is in use.

      Version-Release number of selected component (if applicable): 2.1.0-0

      How reproducible:

      installed SMMR2.1.0 via OpenShift 4.7 operator .
      created any project
      need to give it a special K8s cluster role :
      ```
      cat <<EOF |kubectl apply -f -
      kind: ClusterRole
      apiVersion: rbac.authorization.k8s.io/v1
      metadata:
      name: myApp
      rules:

      • apiGroups: ["security.openshift.io"]
        resources: ["securitycontextconstraints"]
        verbs: ["use"]


      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
      name: myApp
      roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: myApp
      subjects:

      • kind: ServiceAccount
        name: myApp
        namespace: myNamespace

      ```

      The deployment will start but the istio sidecar will crash shouting about permissions:
      ```
      2021-12-07T13:07:54.771204Z info citadelclient Citadel client using custom root cert: istiod-gateways-on-edge-pool.istio-system.svc:15012
      2021-12-07T13:07:54.795262Z info ads All caches have been synced up in 27.892446ms, marking server ready
      2021-12-07T13:07:54.795818Z error sds Failed to set up UDS path: failed to listen on unix socket "./etc/istio/proxy/SDS": listen unix ./etc/istio/proxy/SDS: bind: permission denied
      2021-12-07T13:07:54.795844Z info sds SDS server for workload certificates started, listening on "./etc/istio/proxy/SDS"
      2021-12-07T13:07:54.796016Z info sds Start SDS grpc server
      2021-12-07T13:07:54.796272Z error sds SDS grpc server for workload proxies failed to set up UDS: failed to listen on unix socket "./etc/istio/proxy/SDS": listen unix ./etc/istio/proxy/SDS: bind: permission denied
      2021-12-07T13:07:54.796331Z info xdsproxy Initializing with upstream address "istiod-gateways-on-edge-pool.istio-system.svc:15012" and cluster "Kubernetes"
      2021-12-07T13:07:54.796361Z info dns Starting local udp DNS server at localhost:15053
      2021-12-07T13:07:54.796455Z fatal Failed to start in-process SDSfailed to start xds proxy: failed to listen on unix socket "./etc/istio/proxy/XDS": listen unix ./etc/istio/proxy/XDS: bind: permission denied
      ```

      therefore, the pod is red.
      tried all kind of voodoo actions like https://istio.io/latest/docs/setup/platform-setup/openshift/
      nothing helped.

      tried giving special SCC :
      $ oc adm policy add-scc-to-user ibm-anyuid-hostpath-scc -z [MY_SERVICE_ACCOUNT]...
      $ oc adm policy add-scc-to-user ibm-anyuid-scc -z....
      $ oc adm policy add-scc-to-user ibm-privileged-scc

      nothing helped.

      Actual results:
      istio sidecar is down

      Expected results:
      istio sidecar report container ready

      Additional info:
      the reason for that cluster role is that I wrote a K8s operator which need to create a RoleBinding that uses this role binding ref :`system:openshift:scc:ibm-privileged-scc` .

      RoleBindings:
      RoleRef: APIGroup: "rbac.authorization.k8s.io",
      Kind: "ClusterRole",
      Name: system:openshift:scc:ibm-privileged-scc
      Subject:...

              cewong@redhat.com Cesar Wong
              openshift_jira_bot OpenShift Jira Bot
              Red Hat Employee
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: