Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-14049

OpenShift on OpenStack: Password Rotation of OSP User still leads to unknown authentication failures in Keystone

    XMLWordPrintable

Details

    • ?
    • Important
    • No
    • ShiftStack Sprint 238, ShiftStack Sprint 239, ShiftStack Sprint 240
    • 3
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      * The Manila CSI Driver Operator automatically creates an OpenShift storage class for each available Manila share type. As part of this operation, the Operator queries the Manila API. Previously, the Operator cached the cloud credentials, which resulted in authentication issues when these credentials were rotated. Now, the Operator always uses the latest credentials. (link:https://issues.redhat.com/browse/OCPBUGS-14049[*OCPBUGS-14049*])
      Show
      * The Manila CSI Driver Operator automatically creates an OpenShift storage class for each available Manila share type. As part of this operation, the Operator queries the Manila API. Previously, the Operator cached the cloud credentials, which resulted in authentication issues when these credentials were rotated. Now, the Operator always uses the latest credentials. (link: https://issues.redhat.com/browse/OCPBUGS-14049 [* OCPBUGS-14049 *])
    • Bug Fix
    • Done
    • Customer Escalated

    Description

      Description of problem:

      After all cluster operators have reconciled after the password rotation, we can still see authentication failures in keystone (attached screenshot of splunk query)

      Version-Release number of selected component (if applicable):

      Environment:
- OpenShift 4.12.10 on OpenStack 16
- The cluster is managed via RHACM, but password rotation shall be done via "regular"  OpenShift means.

      How reproducible:

      Rotated the OpenStack credentials according to the documentation [1]

[1] https://docs.openshift.com/container-platform/4.12/authentication/managing_cloud_provider_credentials/cco-mode-passthrough.html#manually-rotating-cloud-creds_cco-mode-passthrough

      Additional info:

      - we can't trace back where these authentication failures come from - they do disappear after a cluster upgrade (so when nodes are rebooted and all pods are restarted which indicates that there's still a component using the old credentials)
- The relevant technical integration points _seem_ to be working though (LBaaS, CSI, Machine API, Swift)

      What is the business impact? Please also provide timeframe information.

      - We cannot rely on splunk monitoring for authentication issues since it's currently constantly showing authentication errors - We cannot be entirely sure that everything works as expected since we don't know the component that doesn't seem to use the new credentials

       

      Attachments

        Issue Links

          blocks

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-17160 OpenShift on OpenStack: Password Rotation of OSP User still leads to unknown authentication failures in Keystone

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-17160 OpenShift on OpenStack: Password Rotation of OSP User still leads to unknown authentication failures in Keystone

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          links to

          GitHub openshift/csi-driver-manila-operator#184: OCPBUGS-14049: Don't cache OpenStack client

          GitHub openshift/csi-driver-manila-operator#187: OCPBUGS-14049: Add secret informer

          GitHub openshift/csi-driver-manila-operator#192: Revert "OCPBUGS-14049: Add secret informer"

          Web Link RHEA-2023:5006 rpm

          Activity

            People

              sfinucan@redhat.com Stephen Finucane
              rhn-support-aprajapa Ashish Prajapati
              Itshak Brown Itshak Brown
              Janine Eichler, Mariya Gokhool
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: