Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-7390

During SNO installation pods failed to get created due to missing openshift.io/sa.scc.uid-range annotation

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Undefined Undefined
    • None
    • 4.12
    • Cluster Version Operator
    • None

      Description of the problem:

      When installing SNO pods are not getting created because their namespace isn't annotated with openshift.io/sa.scc.uid-range

      Eventually, the missing annotation get added but the result is a delay in the installation completion + random and unexpected error events 

      How reproducible:

      100%

       

      Steps to reproduce:

      1. install SNO with bootstrap in place (https://github.com/eranco74/bootstrap-in-place-poc)

      2. check the events log:

      oc get events -A -o custom-columns=FirstSeen:.firstTimestamp,LastSeen:.lastTimestamp,Count:.count,From:.source.component,Type:.type,Reason:.reason,Message:.message | grep "unable to find annotation openshift.io/sa.scc.uid-range"

      3.

      Actual results:

      This is the list of namespaces:

      1.  ** openshift-apiserver
      2.  ** openshift-console
      3.  openshift-console-operator
      4. openshift-controller-manager
      5. openshift-ingress-canary
      6. openshift-ingress
      7. openshift-machine-api
      8. openshift-network-diagnostics
      9. openshift-oauth-apiserver
      10. openshift-route-controller-manager
      11. openshift-service-ca (specifically delay the installation the most https://issues.redhat.com/browse/API-1525)

       

       oc get events -A -o custom-columns=FirstSeen:.firstTimestamp,LastSeen:.lastTimestamp,Count:.count,From:.source.component,Type:.type,Reason:.reason,Message:.message | grep "unable to find annotation openshift.io/sa.scc.uid-range"
2023-02-08T13:27:26Z   2023-02-08T13:27:26Z   1        replicaset-controller                                                                                                                                         Warning   FailedCreate                             Error creating: pods "apiserver-5c9c588976-" is forbidden: error fetching namespace "openshift-apiserver": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:27:36Z   2023-02-08T13:28:38Z   7        replicaset-controller                                                                                                                                         Warning   FailedCreate                             Error creating: pods "apiserver-b8dc7966c-" is forbidden: error fetching namespace "openshift-apiserver": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:38:22Z   2023-02-08T13:39:52Z   10       replicaset-controller                                                                                                                                         Warning   FailedCreate                             Error creating: pods "console-operator-69cf76ffb-" is forbidden: error fetching namespace "openshift-console-operator": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:27:19Z   2023-02-08T13:27:19Z   1        replicaset-controller                                                                                                                                         Warning   FailedCreate                             Error creating: pods "controller-manager-65c889647b-" is forbidden: error fetching namespace "openshift-controller-manager": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:27:38Z   2023-02-08T13:28:39Z   7        replicaset-controller                                                                                                                                         Warning   FailedCreate                             Error creating: pods "controller-manager-666c9bf5dd-" is forbidden: error fetching namespace "openshift-controller-manager": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:27:17Z   2023-02-08T13:27:17Z   1        replicaset-controller                                                                                                                                         Warning   FailedCreate                             Error creating: pods "controller-manager-67dc499947-" is forbidden: error fetching namespace "openshift-controller-manager": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:38:22Z   2023-02-08T13:39:55Z   10       daemonset-controller                                                                                                                                          Warning   FailedCreate                             Error creating: pods "ingress-canary-" is forbidden: error fetching namespace "openshift-ingress-canary": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:27:16Z   2023-02-08T13:28:38Z   8        replicaset-controller                                                                                                                                         Warning   FailedCreate                             Error creating: pods "migrator-74ff46c5c6-" is forbidden: error fetching namespace "openshift-kube-storage-version-migrator": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:25:29Z   2023-02-08T13:28:06Z   15       replicaset-controller                                                                                                                                         Warning   FailedCreate                             Error creating: pods "network-check-source-6b9f7f5bff-" is forbidden: error fetching namespace "openshift-network-diagnostics": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:25:30Z   2023-02-08T13:28:21Z   14       daemonset-controller                                                                                                                                          Warning   FailedCreate                             Error creating: pods "network-check-target-" is forbidden: error fetching namespace "openshift-network-diagnostics": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:27:38Z   2023-02-08T13:28:39Z   7        replicaset-controller                                                                                                                                         Warning   FailedCreate                             Error creating: pods "apiserver-556b68c44b-" is forbidden: error fetching namespace "openshift-oauth-apiserver": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:27:29Z   2023-02-08T13:27:29Z   1        replicaset-controller                                                                                                                                         Warning   FailedCreate                             Error creating: pods "apiserver-dbc4ff9c9-" is forbidden: error fetching namespace "openshift-oauth-apiserver": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:27:17Z   2023-02-08T13:27:28Z   2        replicaset-controller                                                                                                                                         Warning   FailedCreate                             Error creating: pods "route-controller-manager-6b85f59f6f-" is forbidden: error fetching namespace "openshift-route-controller-manager": unable to find annotation openshift.io/sa.scc.uid-range
2023-02-08T13:27:39Z   2023-02-08T13:28:36Z   6        replicaset-controller                                                                                                                                         Warning   FailedCreate                             Error creating: pods "route-controller-manager-6f9f78b955-" is forbidden: error fetching namespace "openshift-route-controller-manager": unable to find annotation openshift.io/sa.scc.uid-range

      Expected results:

      expected the namespace to get the annotation by the time pods need to start.

      The issue goes away if I add the yamls for these namespaces to the manifests directory prior to generating the ignition.

        

            mresvani@redhat.com Michail Resvanis
            ercohen Eran Cohen
            Jia Liu Jia Liu
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: