Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-63238

endpoint rule error - ccoctl in OpenShift v4.21

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • 4.21
    • None
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • Important
    • None
    • Dev
    • None
    • None
    • Proposed
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      Reported by customer:
      
      https://github.com/openshift/cloud-credential-operator/pull/921
      This change removed the overrides for the s3/cf endpoints because it was thought that they were not in use. I believe they were in use for the ccoctl binary that we use them for setting up OIDC in our newly created clusters.
      This morning when we rebuilt our dev environments and ran ccoctl we found the following endpoint rule error:
      
      ccoctl aws create-identity-provider --output-dir X --name X --region us-east-1
      
      2025/09/29 09:25:22 failed to create a bucket to store OpenID Connect configuration: operation error S3: CreateBucket, resolve auth scheme: resolve endpoint: endpoint rule error, Invalid region: region was not a valid DNS name.
      
      
      It can no longer resolve our private endpoint for s3 due to the removed code in the PR. When we rolled back to an older version it worked again.
      
      As the PR alludes If either of these clients become needed in the future, or the ccoctl gains the ability to override the endpoints, then this functionality will need to be sorted out such that the proper endpoint override is being assigned to each client.Would it be possible to "sort out the proper endpoint override" before we upgrade to 4.21? We're currently on 4.18 so we have a long runway to get there.

      Version-Release number of selected component (if applicable):

          

      How reproducible:

          Every time

      Steps to Reproduce:

          1. Run: ccoctl aws create-identity-provider --output-dir X --name dev-X --region us-east-1

      Actual results:

          2025/09/29 09:25:22 failed to create a bucket to store OpenID Connect configuration: operation error S3: CreateBucket, resolve auth scheme: resolve endpoint: endpoint rule error, Invalid region: region was not a valid DNS name.

      Expected results:

      OIDC set up correctly

      Additional info:

          From Jeremiah Stuever:
      I would recommend they use officially released versions that have gone through significantly more quality assurances.
      
      As for the issue at hand, this may be a bug. However, it is very unlikely due to the specified PR. The ccoctl binary doesn't set any endpoints as seen in the following code path:
      The ccoctl aws create-identity-provider command uses an aws client created via newAWSClient().
      https://github.com/openshift/cloud-credential-operator/blob/1df8ad9bdf726e9c8c34437f294aa4e84d1794ed/pkg/cmd/provisioning/aws/create_identity_provider.go#L755The newAWSClient() function creates the client via NewClientFromConfig() while passing an empty string for the endpoint parameter.
      https://github.com/openshift/cloud-credential-operator/blob/1df8ad9bdf726e9c8c34437f294aa4e84d1794ed/pkg/cmd/provisioning/aws/aws.go#L50The NewClientFromConfig() function is where the specified PR made changes. However, it sets the endpoints when the endpoint parameter is not an empty string. Removing the endpoint code for the S3 and CloudFront should not have had any affect in the ccoctl binary because they were always an empty string and therefore never being called.
      https://github.com/openshift/cloud-credential-operator/blob/1df8ad9bdf726e9c8c34437f294aa4e84d1794ed/pkg/aws/client.go#L344-L348It is much more likely that the bug originates from a PR where we migrated from AWS SDK v1 to v2. This merged just a few days prior.
      
      https://github.com/openshift/cloud-credential-operator/pull/905
      
      Please open a bug so we can diagnose and resolve this issue. I suspect we would also want a support case attached to the bug so we can communicate with the customer to troubleshoot the issue.

              jstuever@redhat.com Jeremiah Stuever
              jpollard@redhat.com James Pollard
              None
              None
              Jianping Shu Jianping Shu
              None
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: