-
Bug
-
Resolution: Done-Errata
-
Critical
-
None
-
4.16
-
No
-
Rejected
-
False
-
Description of problem:
Trying to execute https://github.com/openshift-metal3/dev-scripts to deploy an OCP 4.16 or 4.17 cluster (with the same configuration OCP 4.14 and 4.15 are instead working) with: MIRROR_IMAGES=true INSTALLER_PROXY=true the bootstrap process fails with: level=debug msg= baremetalhost resource not yet available, will retry level=debug msg= baremetalhost resource not yet available, will retry level=info msg= baremetalhost: ostest-master-0: uninitialized level=info msg= baremetalhost: ostest-master-0: registering level=info msg= baremetalhost: ostest-master-1: uninitialized level=info msg= baremetalhost: ostest-master-1: registering level=info msg= baremetalhost: ostest-master-2: uninitialized level=info msg= baremetalhost: ostest-master-2: registering level=info msg= baremetalhost: ostest-master-1: inspecting level=info msg= baremetalhost: ostest-master-2: inspecting level=info msg= baremetalhost: ostest-master-0: inspecting E0514 12:16:51.985417 89709 reflector.go:147] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to watch *unstructured.Unstructured: Get "https://api.ostest.test.metalkube.org:6443/apis/metal3.io/v1alpha1/namespaces/openshift-machine-api/baremetalhosts?allowWatchBookmarks=true&resourceVersion=5466&timeoutSeconds=547&watch=true": Service Unavailable W0514 12:16:52.979254 89709 reflector.go:539] k8s.io/client-go/tools/watch/informerwatcher.go:146: failed to list *unstructured.Unstructured: Get "https://api.ostest.test.metalkube.org:6443/apis/metal3.io/v1alpha1/namespaces/openshift-machine-api/baremetalhosts?resourceVersion=5466": Service Unavailable E0514 12:16:52.979293 89709 reflector.go:147] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to watch *unstructured.Unstructured: failed to list *unstructured.Unstructured: Get "https://api.ostest.test.metalkube.org:6443/apis/metal3.io/v1alpha1/namespaces/openshift-machine-api/baremetalhosts?resourceVersion=5466": Service Unavailable E0514 12:37:01.927140 89709 reflector.go:147] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to watch *unstructured.Unstructured: Get "https://api.ostest.test.metalkube.org:6443/apis/metal3.io/v1alpha1/namespaces/openshift-machine-api/baremetalhosts?allowWatchBookmarks=true&resourceVersion=7800&timeoutSeconds=383&watch=true": Service Unavailable W0514 12:37:03.173425 89709 reflector.go:539] k8s.io/client-go/tools/watch/informerwatcher.go:146: failed to list *unstructured.Unstructured: Get "https://api.ostest.test.metalkube.org:6443/apis/metal3.io/v1alpha1/namespaces/openshift-machine-api/baremetalhosts?resourceVersion=7800": Service Unavailable E0514 12:37:03.173473 89709 reflector.go:147] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to watch *unstructured.Unstructured: failed to list *unstructured.Unstructured: Get "https://api.ostest.test.metalkube.org:6443/apis/metal3.io/v1alpha1/namespaces/openshift-machine-api/baremetalhosts?resourceVersion=7800": Service Unavailable level=debug msg=Fetching Bootstrap SSH Key Pair... level=debug msg=Loading Bootstrap SSH Key Pair... it looks like up to a certain point https://api.ostest.test.metalkube.org:6443 was reachable but then for some reason it started failing because its not using the proxy or is and it shouldn't be (???) The 3 master nodes are reported as: [root@ipi-ci-op-0qigcrln-b54ee-1790684582253694976 home]# oc get baremetalhosts -A NAMESPACE NAME STATE CONSUMER ONLINE ERROR AGE openshift-machine-api ostest-master-0 inspecting ostest-bbhxb-master-0 true inspection error 24m openshift-machine-api ostest-master-1 inspecting ostest-bbhxb-master-1 true inspection error 24m openshift-machine-api ostest-master-2 inspecting ostest-bbhxb-master-2 true inspection error 24m With something like: status: errorCount: 5 errorMessage: 'Failed to inspect hardware. Reason: unable to start inspection: Validation of image href http://0.0.0.0:8084/34427934-f1a6-48d6-9666-66872eec9ba2 failed, reason: Got HTTP code 503 instead of 200 in response to HEAD request.' errorType: inspection error on their status
Version-Release number of selected component (if applicable):
4.16, 4.17
How reproducible:
100%
Steps to Reproduce:
1. Try to create an OCP 4.16 cluster with dev-scrips with IP_STACK=v4, MIRROR_IMAGES=true and INSTALLER_PROXY=true 2. 3.
Actual results:
level=info msg= baremetalhost: ostest-master-0: inspecting E0514 12:16:51.985417 89709 reflector.go:147] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to watch *unstructured.Unstructured: Get "https://api.ostest.test.metalkube.org:6443/apis/metal3.io/v1alpha1/namespaces/openshift-machine-api/baremetalhosts?allowWatchBookmarks=true&resourceVersion=5466&timeoutSeconds=547&watch=true": Service Unavailable
Expected results:
Successful deployment
Additional info:
I'm using IP_STACK=v4, MIRROR_IMAGES=true and INSTALLER_PROXY=true with the same configuration (MIRROR_IMAGES=true and INSTALLER_PROXY=true) OCP 4.14 and OCP 4.15 are working. When removing INSTALLER_PROXY=true, OCP 4.16 is also working. I'm going to attach bootstrap gather logs
- blocks
-
CNV-41570 Disconnected CI job is permanently broken
- Closed
- is cloned by
-
OCPBUGS-35818 [4.16] dev-scripts fails bootstrapping OCP 4.16 and greater with MIRROR_IMAGES=true AND INSTALLER_PROXY=true
- Closed
- is depended on by
-
OCPBUGS-35818 [4.16] dev-scripts fails bootstrapping OCP 4.16 and greater with MIRROR_IMAGES=true AND INSTALLER_PROXY=true
- Closed
- is duplicated by
-
OCPBUGS-35751 Cannot deploy IPv4 only, connected hub with proxy - inspection failure
- Closed
- links to
-
RHEA-2024:3718 OpenShift Container Platform 4.17.z bug fix update