Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-4173

vsphere-problem-detector should re-check passwords after change

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Normal Normal
    • None
    • 4.12
    • Storage / Operators
    • None
    • Moderate
    • None
    • False
    • Hide

      None

      Show
      None
    • N/A
    • Release Note Not Required

      Description of problem:

      When user changes vSphere configuration or credentials (username, password, vCenter address, ...), vsphere-problem-detector should re-check them quickly and not wait for a periodic re-check that can happen after 1 hour.

       

      Version-Release number of selected component (if applicable): 4.13.0-0.nightly-2022-11-25-204445, but all previous versions are probably affected too.

      How reproducible: Always

      Steps to Reproduce:

      1. Install a cluster on vSphere (with valid credentials)
      2. Configure a bad username / password
      3. See that ClusterCSIDriver for vSphere CSI driver gets Degraded in ~2 minutes (that's vsphere-csi-driver-operator, it's quick)
      4. Wait until vsphere-problem-detector realizes it's a bad password (could take up to 1 hour):
        1. See that oc get storage -o yaml shows VSphereProblemDetectorControllerAvailable as "True" with message failed to connect to vcenter.XYZ: Cannot complete login due to an incorrect user name or password
        2. See that VSphereOpenshiftConnectionFailure alert is firing (or at least Pending)
      5. Configure correct username/password

      Actual results:

      It takes up to 1 hour for vsphere-problem-detector to re-check the password

      Expected results:

      vsphere-problem-detector re-checks the new password in few minutes (due to leader election it can't be instant). The alert + VSphereProblemDetectorControllerAvailable conditions are cleared in 5 minutes max.

            [OCPBUGS-4173] vsphere-problem-detector should re-check passwords after change

            Errata Tool added a comment -

            Since the problem described in this issue should be resolved in a recent advisory, it has been closed.

            For information on the advisory (Important: OpenShift Container Platform 4.13.0 security update), and where to find the updated files, follow the link below.

            If the solution does not work for you, open a new bug report.
            https://access.redhat.com/errata/RHSA-2023:1326

            Errata Tool added a comment - Since the problem described in this issue should be resolved in a recent advisory, it has been closed. For information on the advisory (Important: OpenShift Container Platform 4.13.0 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2023:1326

            Wei Duan added a comment -

            Verified pass on 4.13.0-0.nightly-2022-12-08-005541

            Wei Duan added a comment - Verified pass on 4.13.0-0.nightly-2022-12-08-005541

            Wei Duan added a comment -

            Just a few minutes:

            1. $ oc get co storage -o yaml

                message: 'VSphereCSIDriverOperatorCRDegraded: VMwareVSphereControllerDegraded:
                  error logging into vcenter: ServerFaultCode: Cannot complete login due to an
                  incorrect user name or password.'
                reason: VSphereCSIDriverOperatorCR_VMwareVSphereController_SyncError
                status: "True"
                type: Degraded

             

             

            2. The alert is pending:

            {
              "labels":

            {     "alertname": "VSphereOpenshiftConnectionFailure",     "container": "vsphere-problem-detector-operator",     "endpoint": "vsphere-metrics",     "instance": "10.130.0.59:8444",     "job": "vsphere-problem-detector-metrics",     "namespace": "openshift-cluster-storage-operator",     "pod": "vsphere-problem-detector-operator-7dc9dd44d-pv64g",     "reason": "InvalidCredentials",     "service": "vsphere-problem-detector-metrics",     "severity": "warning"   }

            ,
              "annotations":

            {     "description": "vsphere-problem-detector cannot access vCenter. As consequence, other OCP components,\nsuch as storage or machine API, may not be able to access vCenter too and provide\ntheir services. Detailed error message can be found in Available condition of\nClusterOperator \"storage\", either in console\n(Administration -> Cluster settings -> Cluster operators tab -> storage) or on\ncommand line: oc get clusteroperator storage -o jsonpath='\{.status.conditions[?(@.type==\"Available\")].message}

            '\n",
                "summary": "vsphere-problem-detector is unable to connect to vSphere vCenter."
              },
              "state": "pending",
              "activeAt": "2022-12-08T10:45:15.495869306Z",
              "value": "1e+00"
            }

            Wei Duan added a comment - Just a few minutes: 1. $ oc get co storage -o yaml     message: 'VSphereCSIDriverOperatorCRDegraded: VMwareVSphereControllerDegraded:       error logging into vcenter: ServerFaultCode: Cannot complete login due to an       incorrect user name or password.'     reason: VSphereCSIDriverOperatorCR_VMwareVSphereController_SyncError     status: "True"     type: Degraded     2. The alert is pending: {   "labels": {     "alertname": "VSphereOpenshiftConnectionFailure",     "container": "vsphere-problem-detector-operator",     "endpoint": "vsphere-metrics",     "instance": "10.130.0.59:8444",     "job": "vsphere-problem-detector-metrics",     "namespace": "openshift-cluster-storage-operator",     "pod": "vsphere-problem-detector-operator-7dc9dd44d-pv64g",     "reason": "InvalidCredentials",     "service": "vsphere-problem-detector-metrics",     "severity": "warning"   } ,   "annotations": {     "description": "vsphere-problem-detector cannot access vCenter. As consequence, other OCP components,\nsuch as storage or machine API, may not be able to access vCenter too and provide\ntheir services. Detailed error message can be found in Available condition of\nClusterOperator \"storage\", either in console\n(Administration -> Cluster settings -> Cluster operators tab -> storage) or on\ncommand line: oc get clusteroperator storage -o jsonpath='\{.status.conditions[?(@.type==\"Available\")].message} '\n",     "summary": "vsphere-problem-detector is unable to connect to vSphere vCenter."   },   "state": "pending",   "activeAt": "2022-12-08T10:45:15.495869306Z",   "value": "1e+00" }

              rhn-engineering-jsafrane Jan Safranek
              rhn-engineering-jsafrane Jan Safranek
              Wei Duan Wei Duan
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: