Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-14049

OpenShift on OpenStack: Password Rotation of OSP User still leads to unknown authentication failures in Keystone

XMLWordPrintable

    • -
    • Important
    • No
    • ShiftStack Sprint 238, ShiftStack Sprint 239, ShiftStack Sprint 240
    • 3
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      * The Manila CSI Driver Operator automatically creates an OpenShift storage class for each available Manila share type. As part of this operation, the Operator queries the Manila API. Previously, the Operator cached the cloud credentials, which resulted in authentication issues when these credentials were rotated. Now, the Operator always uses the latest credentials. (link:https://issues.redhat.com/browse/OCPBUGS-14049[*OCPBUGS-14049*])
      Show
      * The Manila CSI Driver Operator automatically creates an OpenShift storage class for each available Manila share type. As part of this operation, the Operator queries the Manila API. Previously, the Operator cached the cloud credentials, which resulted in authentication issues when these credentials were rotated. Now, the Operator always uses the latest credentials. (link: https://issues.redhat.com/browse/OCPBUGS-14049 [* OCPBUGS-14049 *])
    • Bug Fix
    • Done
    • Customer Escalated

      Description of problem:

      After all cluster operators have reconciled after the password rotation, we can still see authentication failures in keystone (attached screenshot of splunk query)

      Version-Release number of selected component (if applicable):

      Environment:
- OpenShift 4.12.10 on OpenStack 16
- The cluster is managed via RHACM, but password rotation shall be done via "regular"  OpenShift means.

      How reproducible:

      Rotated the OpenStack credentials according to the documentation [1]

[1] https://docs.openshift.com/container-platform/4.12/authentication/managing_cloud_provider_credentials/cco-mode-passthrough.html#manually-rotating-cloud-creds_cco-mode-passthrough

      Additional info:

      - we can't trace back where these authentication failures come from - they do disappear after a cluster upgrade (so when nodes are rebooted and all pods are restarted which indicates that there's still a component using the old credentials)
- The relevant technical integration points _seem_ to be working though (LBaaS, CSI, Machine API, Swift)

      What is the business impact? Please also provide timeframe information.

      - We cannot rely on splunk monitoring for authentication issues since it's currently constantly showing authentication errors - We cannot be entirely sure that everything works as expected since we don't know the component that doesn't seem to use the new credentials

       

              sfinucan@redhat.com Stephen Finucane
              rhn-support-aprajapa Ashish Prajapati
              Itshak Brown Itshak Brown
              Janine Eichler, Mariya Gokhool
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

                Created:
                Updated:
                Resolved: