Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-59152

cloud-network-config-controller Crashes on Azure Due to Newline Character in Credentials

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

          During an OCP IPI/UPI installation on Azure, the cloud-network-config-controller pod within the openshift-cloud-network-config-controller project goes into a CrashLoopBackOff state.

      Error Log: The pod logs show a fatal error indicating an invalid Azure Tenant ID:

      2025-07-07T13:57:54.963969859Z F0707 13:57:54.963938       1 main.go:143] Error building cloud provider client, err: invalid tenantID. You can locate your tenantID by following the instructions listed here: https://docs.microsoft.com/partner-center/find-ids-and-domain-names

      Root Cause:
      The issue stems from a newline character present in the base64-encoded cloud-credentials secret (Cg==). When the controller decodes and uses this secret, the Azure API rejects the tenant ID because it contains this extraneous character.

      Workaround: Manually editing the cloud-credentials secret to remove the newline character resolves the issue, and the pod comes into a Running state.

      Analysis

      A significant observation is that only the cloud-network-config-controller fails, while other Cluster Operators (COs) using the same cloud-credentials secret function correctly. This strongly suggests an inconsistency in how credentials are handled across different OpenShift components.

      • Hypothesis on openshift-installer: The openshift-installer likely trims whitespace and newline characters from credentials during the initial validation phase. This allows it to authenticate with Azure successfully and complete the cluster installation, masking the malformed data in the secret.
      • Hypothesis on cloud-network-config-controller: Post-installation, this specific controller reads the secret data "as-is". It does not sanitize the input, so it passes the tenant ID with the embedded newline character directly to the Azure API, causing the authentication to fail.

      Version-Release number of selected component (if applicable):

          4.17,4.18

      Actual results:

          The cloud-network-config-controller is not robust against malformed data in the cloud-credentials secret. It reads the data literally, causing authentication to fail and the pod to enter a CrashLoopBackOff loop, which can disrupt network configuration management in the cluster.

      Expected results:

          The cloud-network-config-controller should be resilient to common input formatting issues like leading/trailing whitespace or newlines in the cloud-credentials secret. It should sanitize the credential data before use, ensuring stable operation, consistent with how the openshift-installer appears to handle the same data.

      By addressing this inconsistency, future installations will be more resilient to common user input errors, preventing post-installation component failures.

       

              Unassigned Unassigned
              rhn-support-dpateriy Divyam Pateriya
              None
              None
              Gaoyun Pei Gaoyun Pei
              None
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: