Description of problem:

Installation fails with ingress operator degraded and *.apps DNS record is not created.

[acapriot@acapriot-thinkpadp1gen4i ibmcloud]$ oc --kubeconfig=install-dir/auth/kubeconfig  get co ingress

NAME  VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE

ingress         False   True      True   48m The "default" ingress controller reports Available=False: IngressControllerUnavailable: One or more status conditions indicate unavailable: DNSReady=False (NoZones: The record isn't present in any zones.)

I added them manually.

$ oc --kubeconfig=install-dir/auth/kubeconfig  create -f install-dir/ingresscredential/openshift-ingress-operator-cloud-credentials-credentials.yaml

Workaround: Remove the --included --install-config parameters to extract all the manifests in a tmp directory:

$ oc adm release extract --from=$RELEASE_IMAGE --credentials-requests --to=/tmp/allcloudcred

warning: if you intend to pass CredentialsRequests to ccoctl, you should use --included to filter out requests that your cluster is not expected to need.

Extracted release payload created at 2024-06-27T12:33:04Z

 

Edit file 0000_50_cluster-ingress-operator_00-ingress-credentials-request.yaml and remove all entries except IBM ones:

$ cat install-dir/cloudcredential/0000_50_cluster-ingress-operator_00-ingress-credentials-request.yaml

—

apiVersion: cloudcredential.openshift.io/v1

kind: CredentialsRequest

metadata:

  annotations:

capability.openshift.io/name: CloudCredential+Ingress

  name: openshift-ingress-ibmcloud

  namespace: openshift-cloud-credential-operator

spec:

  providerSpec:

apiVersion: cloudcredential.openshift.io/v1

kind: IBMCloudProviderSpec

policies:

• attributes:

   - name: serviceName

     value: internet-svcs

   roles:

   - crn:v1:bluemix:public:iam::::serviceRole:Manager

   - crn:v1:bluemix:public:iam::::serviceRole:Reader

   - crn:v1:bluemix:public:iam::::serviceRole:Writer

• attributes:

   - name: serviceName

     value: dns-svcs

   roles:

   - crn:v1:bluemix:public:iam::::serviceRole:Manager

   - crn:v1:bluemix:public:iam::::serviceRole:Reader

   - crn:v1:bluemix:public:iam::::serviceRole:Writer

  secretRef:

name: cloud-credentials

namespace: openshift-ingress-operator

—

apiVersion: cloudcredential.openshift.io/v1

kind: CredentialsRequest

metadata:

  annotations:

capability.openshift.io/name: CloudCredential+Ingress

  name: openshift-ingress-powervs

  namespace: openshift-cloud-credential-operator

spec:

  providerSpec:

apiVersion: cloudcredential.openshift.io/v1

kind: IBMCloudPowerVSProviderSpec

policies:

• attributes:

   - name: serviceName

     value: internet-svcs

   roles:

   - crn:v1:bluemix:public:iam::::serviceRole:Manager

   - crn:v1:bluemix:public:iam::::serviceRole:Reader

   - crn:v1:bluemix:public:iam::::serviceRole:Writer

• attributes:

   - name: serviceName

     value: dns-svcs

   roles:

   - crn:v1:bluemix:public:iam::::serviceRole:Manager

   - crn:v1:bluemix:public:iam::::serviceRole:Reader

   - crn:v1:bluemix:public:iam::::serviceRole:Writer

  secretRef:

name: cloud-credentials

namespace: openshift-ingress-operator

Copy the file in install-dir/cloudcredential. 

Create the service ID for each credential request, assign the policies defined, create an API key, and generate the secret:

$ ccoctl.rhel9 ibmcloud create-service-id --credentials-requests-dir=install-dir/cloudcredential/ --name=hackathon-416-acapriot --output-dir=install-dir/

[..]
2024/07/07 18:23:21 Saved credentials configuration to: install-dir/manifests/openshift-cloud-controller-manager-ibm-cloud-credentials-credentials.yaml

2024/07/07 18:23:21 Saved credentials configuration to: install-dir/manifests/openshift-machine-api-ibmcloud-credentials-credentials.yaml

2024/07/07 18:23:21 Saved credentials configuration to: install-dir/manifests/openshift-image-registry-installer-cloud-credentials-credentials.yaml

2024/07/07 18:23:21 Saved credentials configuration to: install-dir/manifests/openshift-ingress-operator-cloud-credentials-credentials.yaml

2024/07/07 18:23:21 Saved credentials configuration to: install-dir/manifests/openshift-ingress-operator-cloud-credentials-credentials.yaml

2024/07/07 18:23:21 Saved credentials configuration to: install-dir/manifests/openshift-cluster-csi-drivers-ibm-cloud-credentials-credentials.yaml

After adding the missing credential the installation succeeded.

Version-Release number of selected component (if applicable):

4.16.1

How reproducible:

Install Openshift 4.16.1 IPI on IBM Cloud

Steps to Reproduce:

$ export IC_API_KEY=<api_key>
$ ./openshift-install create install-config --dir=install-dir --log-level=debug
$ ./openshift-install create manifests --dir=install-dir --log-level=debug
$ ./openshift-install create cluster --dir=install-dir --log-level=debug

Actual results:

    Installation fails.

Expected results:

    Installation completed successfully

Additional info:

    See https://docs.google.com/document/d/1qd8OtYWTuTupABYlAd4Vv1Xd-NVVOaVQPyi1GtnqdXw/edit#heading=h.w2mrifn5lhns