Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-14412

[4.10]keda-admission, keda-metrics-apiserver,keda-operator pods fail to get brought up due to pod security

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Major Major
    • None
    • premerge
    • Pod Autoscaler
    • None
    • None
    • 2
    • OCPNODE Sprint 237 (Green)
    • 1
    • False
    • Hide

      None

      Show
      None

      Description of problem:
      [4.10]keda-admission, keda-metrics-apiserver,keda-operator pods fail to get brought up due to pod security

      Version-Release number of selected component (if applicable):
      CMA 2.10.1-251 
      OCP 4.10.61

      How reproducible:
      100%

      Steps to Reproduce:
      1. Instal CMA Keda 2.10.1-251  from stage on a 4.10 cluster
      2. Create KedaController with default config
      3. Check the pods/replicaset in -n openshift-keda

      Actual results:

      % oc get replicaset
      NAME                                            DESIRED   CURRENT   READY   AGE
      custom-metrics-autoscaler-operator-79b9f99fdc   1         1         1       10m
      keda-admission-68b7ff8447                       1         0         0       9m43s
      keda-metrics-apiserver-789978c5d9               1         0         0       9m43s
      keda-operator-5977d7c798                        1         0         0       9m43s

      Expected results:
      all pod in openshift-keda are up

      Additional info:
      Issue only happens on 4.10

      % oc describe replicaset keda-admission-68b7ff8447
      Name:           keda-admission-68b7ff8447
      Namespace:      openshift-keda
      Selector:       app=keda-admission-webhooks,pod-template-hash=68b7ff8447
      Labels:         app=keda-admission-webhooks
                      name=keda-admission-webhooks
                      pod-template-hash=68b7ff8447
      Annotations:    deployment.kubernetes.io/desired-replicas: 1
                      deployment.kubernetes.io/max-replicas: 2
                      deployment.kubernetes.io/revision: 1
                      manifestival: new
      Controlled By:  Deployment/keda-admission
      Replicas:       0 current / 1 desired
      Pods Status:    0 Running / 0 Waiting / 0 Succeeded / 0 Failed
      Pod Template:
        Labels:           app=keda-admission-webhooks
                          name=keda-admission-webhooks
                          pod-template-hash=68b7ff8447
        Service Account:  keda-operator
        Containers:
         keda-admission-webhooks:
          Image:       registry.redhat.io/custom-metrics-autoscaler/custom-metrics-autoscaler-admission-webhooks-rhel8@sha256:7acff4bc838c671b10d5a9ff9aa277aad9c2bba9464b42ebf45a6bd82b2433e7
          Ports:       9443/TCP, 8080/TCP
          Host Ports:  0/TCP, 0/TCP
          Command:
            /keda-admission-webhooks
          Args:
            --zap-log-level=info
            --zap-encoder=console
            --zap-time-encoding=rfc3339
          Limits:
            cpu:     1
            memory:  1000Mi
          Requests:
            cpu:      100m
            memory:   100Mi
          Liveness:   http-get http://:8081/healthz delay=25s timeout=1s period=10s #success=1 #failure=3
          Readiness:  http-get http://:8081/readyz delay=20s timeout=1s period=10s #success=1 #failure=3
          Environment:
            POD_NAMESPACE:               (v1:metadata.namespace)
            WATCH_NAMESPACE:
            KEDA_HTTP_DEFAULT_TIMEOUT:
          Mounts:
            /certs from certificates (ro)
        Volumes:
         certificates:
          Type:                Projected (a volume that contains injected data from multiple sources)
          SecretName:          keda-admission-webhooks-certs
          SecretOptionalName:  <nil>
          ConfigMapName:       keda-ocp-cabundle
          ConfigMapOptional:   <nil>
      Conditions:
        Type             Status  Reason
        ----             ------  ------
        ReplicaFailure   True    FailedCreate
      Events:
        Type     Reason        Age                    From                   Message
        ----     ------        ----                   ----                   -------
        Warning  FailedCreate  9m55s                  replicaset-controller  Error creating: pods "keda-admission-68b7ff8447-shpmp" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/keda-admission-webhooks: Forbidden: seccomp may not be set provider "anyuid": Forbidden: not usable by user or serviceaccount provider "nonroot": Forbidden: not usable by user or serviceaccount provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount provider "hostnetwork": Forbidden: not usable by user or serviceaccount provider "hostaccess": Forbidden: not usable by user or serviceaccount provider "node-exporter": Forbidden: not usable by user or serviceaccount provider "privileged": Forbidden: not usable by user or serviceaccount]
        Warning  FailedCreate  9m55s                  replicaset-controller  Error creating: pods "keda-admission-68b7ff8447-mtm6v" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/keda-admission-webhooks: Forbidden: seccomp may not be set provider "anyuid": Forbidden: not usable by user or serviceaccount provider "nonroot": Forbidden: not usable by user or serviceaccount provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount provider "hostnetwork": Forbidden: not usable by user or serviceaccount provider "hostaccess": Forbidden: not usable by user or serviceaccount provider "node-exporter": Forbidden: not usable by user or serviceaccount provider "privileged": Forbidden: not usable by user or serviceaccount]
        Warning  FailedCreate  9m54s                  replicaset-controller  Error creating: pods "keda-admission-68b7ff8447-tsjv4" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/keda-admission-webhooks: Forbidden: seccomp may not be set provider "anyuid": Forbidden: not usable by user or serviceaccount provider "nonroot": Forbidden: not usable by user or serviceaccount provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount provider "hostnetwork": Forbidden: not usable by user or serviceaccount provider "hostaccess": Forbidden: not usable by user or serviceaccount provider "node-exporter": Forbidden: not usable by user or serviceaccount provider "privileged": Forbidden: not usable by user or serviceaccount]
        Warning  FailedCreate  9m54s                  replicaset-controller  Error creating: pods "keda-admission-68b7ff8447-r5r7j" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/keda-admission-webhooks: Forbidden: seccomp may not be set provider "anyuid": Forbidden: not usable by user or serviceaccount provider "nonroot": Forbidden: not usable by user or serviceaccount provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount provider "hostnetwork": Forbidden: not usable by user or serviceaccount provider "hostaccess": Forbidden: not usable by user or serviceaccount provider "node-exporter": Forbidden: not usable by user or serviceaccount provider "privileged": Forbidden: not usable by user or serviceaccount]
        Warning  FailedCreate  9m54s                  replicaset-controller  Error creating: pods "keda-admission-68b7ff8447-xgwtg" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/keda-admission-webhooks: Forbidden: seccomp may not be set provider "anyuid": Forbidden: not usable by user or serviceaccount provider "nonroot": Forbidden: not usable by user or serviceaccount provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount provider "hostnetwork": Forbidden: not usable by user or serviceaccount provider "hostaccess": Forbidden: not usable by user or serviceaccount provider "node-exporter": Forbidden: not usable by user or serviceaccount provider "privileged": Forbidden: not usable by user or serviceaccount]
        Warning  FailedCreate  9m54s                  replicaset-controller  Error creating: pods "keda-admission-68b7ff8447-9x9kc" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/keda-admission-webhooks: Forbidden: seccomp may not be set provider "anyuid": Forbidden: not usable by user or serviceaccount provider "nonroot": Forbidden: not usable by user or serviceaccount provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount provider "hostnetwork": Forbidden: not usable by user or serviceaccount provider "hostaccess": Forbidden: not usable by user or serviceaccount provider "node-exporter": Forbidden: not usable by user or serviceaccount provider "privileged": Forbidden: not usable by user or serviceaccount]
        Warning  FailedCreate  9m54s                  replicaset-controller  Error creating: pods "keda-admission-68b7ff8447-bclr8" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/keda-admission-webhooks: Forbidden: seccomp may not be set provider "anyuid": Forbidden: not usable by user or serviceaccount provider "nonroot": Forbidden: not usable by user or serviceaccount provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount provider "hostnetwork": Forbidden: not usable by user or serviceaccount provider "hostaccess": Forbidden: not usable by user or serviceaccount provider "node-exporter": Forbidden: not usable by user or serviceaccount provider "privileged": Forbidden: not usable by user or serviceaccount]
        Warning  FailedCreate  9m54s                  replicaset-controller  Error creating: pods "keda-admission-68b7ff8447-7wtv8" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/keda-admission-webhooks: Forbidden: seccomp may not be set provider "anyuid": Forbidden: not usable by user or serviceaccount provider "nonroot": Forbidden: not usable by user or serviceaccount provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount provider "hostnetwork": Forbidden: not usable by user or serviceaccount provider "hostaccess": Forbidden: not usable by user or serviceaccount provider "node-exporter": Forbidden: not usable by user or serviceaccount provider "privileged": Forbidden: not usable by user or serviceaccount]
        Warning  FailedCreate  9m53s                  replicaset-controller  Error creating: pods "keda-admission-68b7ff8447-nnxp9" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/keda-admission-webhooks: Forbidden: seccomp may not be set provider "anyuid": Forbidden: not usable by user or serviceaccount provider "nonroot": Forbidden: not usable by user or serviceaccount provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount provider "hostnetwork": Forbidden: not usable by user or serviceaccount provider "hostaccess": Forbidden: not usable by user or serviceaccount provider "node-exporter": Forbidden: not usable by user or serviceaccount provider "privileged": Forbidden: not usable by user or serviceaccount]
        Warning  FailedCreate  4m27s (x8 over 9m52s)  replicaset-controller  (combined from similar events): Error creating: pods "keda-admission-68b7ff8447-9qtvx" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/keda-admission-webhooks: Forbidden: seccomp may not be set provider "anyuid": Forbidden: not usable by user or serviceaccount provider "nonroot": Forbidden: not usable by user or serviceaccount provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount provider "hostnetwork": Forbidden: not usable by user or serviceaccount provider "hostaccess": Forbidden: not usable by user or serviceaccount provider "node-exporter": Forbidden: not usable by user or serviceaccount provider "privileged": Forbidden: not usable by user or serviceaccount]

              joelsmith.redhat Joel Smith
              rhn-support-weinliu Weinan Liu
              Weinan Liu Weinan Liu
              Zbyněk Roubalík (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: