Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-65648

NooBaa pods were unexpectedly deleted and rescheduled on a non-storage node, causing access issues. The pods later self-recovered, likely after API access was restored on the original storage node

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • 4.16
    • 4.16
    • openshift-apiserver
    • None
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • Important
    • None
    • x86_64
    • Production
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The NooBaa components (noobaa-core-0, noobaa-operator, and noobaa-db-pg-0) were unexpectedly deleted from the storage node after the kubelet received 403 Forbidden errors from the API server when trying to access Secrets and ConfigMaps in the openshift-storage namespace.

      As a result of this authorization failure, the kubelet removed the pods. Immediately afterward, the NooBaa pods were rescheduled on a non-storage node, despite the existence of storage-specific taints. This violated normal placement expectations for OpenShift Data Foundation components. Eventually, after the storage node recovered its ability to communicate with the API, the NooBaa pods were automatically rescheduled back to appropriate storage nodes.

      This unexpected eviction and temporary misplacement of NooBaa caused a short-term service outage, affecting image pulls and storage access cluster-wide.

      The entire behavior was self-recovered after some minutes, without manual intervention, but its root cause (403 errors, incorrect pod placement, and taint bypass) requires further validation by Engineering.

              Unassigned Unassigned
              rhn-support-ravellan Ronald Avellaneda
              None
              None
              Rahul Gangwar Rahul Gangwar
              None
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: