Details
-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
4.13
-
Sprint 235, Sprint 237, Sprint 236
-
3
-
Rejected
-
False
-
Description
Description of problem:
IPI Install failed, in .openshift_install.log level=info msg="Waiting up to 40m0s (until 2:15AM) for the cluster at https://api.ci-op-bv13g48p-4fcb7.qe.azure.devcluster.openshift.com:6443 to initialize..."level=error msg="Attempted to gather ClusterOperator status after installation failure: listing ClusterOperator objects: Get \"https://api.ci-op-bv13g48p-4fcb7.qe.azure.devcluster.openshift.com:6443/apis/config.openshift.io/v1/clusteroperators\": dial tcp 104.43.248.57:6443: i/o timeout"
Version-Release number of selected component (if applicable):
4.13.0-0.nightly-2023-01-31-072358
How reproducible:
sometimes
Steps to Reproduce:
1. IPI install failed 2. check the .openshift_install.log 3. check the lb of the resource groups
Actual results:
in .openshift_install.log level=info msg="Waiting up to 40m0s (until 2:15AM) for the cluster at https://api.ci-op-bv13g48p-4fcb7.qe.azure.devcluster.openshift.com:6443 to initialize..."level=error msg="Attempted to gather ClusterOperator status after installation failure: listing ClusterOperator objects: Get \"https://api.ci-op-bv13g48p-4fcb7.qe.azure.devcluster.openshift.com:6443/apis/config.openshift.io/v1/clusteroperators\": dial tcp 104.43.248.57:6443: i/o timeout"Version: 4.13.0-0.nightly-2023-01-31-072358
After install failed, checked the address-pool of the lb info
https://gcsweb-qe-private-deck-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/qe-private-deck/logs/periodic-ci-openshift-openshift-tests-private-release-4.13-amd64-nightly-azure-ipi-fips-p3-f28-destructive/1620585883407224832/artifacts/azure-ipi-fips-p3-f28-destructive/gather-azure-resource/build-log.txt
No master was added into the backend pool of the public LB, which caused the API was not available
Running Command: az network lb list --resource-group ci-op-bv13g48p-4fcb7-s8dbc-rg -o tsv 1 W/"bf151b74-1b0b-462d-a605-711c752a755f" None 2 /subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ci-op-bv13g48p-4fcb7-s8dbc-rg/providers/Microsoft.Network/loadBalancers/ci-op-bv13g48p-4fcb7-s8dbc 0 0 3 centralus ci-op-bv13g48p-4fcb7-s8dbc 0 3 Succeeded ci-op-bv13g48p-4fcb7-s8dbc-rg c9627793-8fdf-4a8b-8e5e-137b97b3fade Microsoft.Network/loadBalancers 1 W/"b8606534-0fcc-440c-84ae-4a4b41d81082" None 1 /subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ci-op-bv13g48p-4fcb7-s8dbc-rg/providers/Microsoft.Network/loadBalancers/ci-op-bv13g48p-4fcb7-s8dbc-internal 0 0 2 centralus ci-op-bv13g48p-4fcb7-s8dbc-internal 0 2 Succeeded ci-op-bv13g48p-4fcb7-s8dbc-rg 78c70add-65ac-46aa-aee1-7458ed1a800e Microsoft.Network/loadBalancers Running Command: az network lb address-pool address list --lb-name ci-op-bv13g48p-4fcb7-s8dbc --pool-name ci-op-bv13g48p-4fcb7-s8dbc --resource-group ci-op-bv13g48p-4fcb7-s8dbc-rg -o table WARNING: Command group 'network lb address-pool address' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus Name ResourceGroup ------------------------------------ ----------------------------- 0ceee896-a1f2-4358-ab9b-bd1006047d3e ci-op-bv13g48p-4fcb7-s8dbc-rg af305d55-4e58-47b2-8324-31b3aab99807 ci-op-bv13g48p-4fcb7-s8dbc-rg c5b44cb8-01ba-4159-9bc1-702de14343d6 ci-op-bv13g48p-4fcb7-s8dbc-rg Running Command: az network lb address-pool address list --lb-name ci-op-bv13g48p-4fcb7-s8dbc-internal --pool-name ci-op-bv13g48p-4fcb7-s8dbc --resource-group ci-op-bv13g48p-4fcb7-s8dbc-rg -o table WARNING: Command group 'network lb address-pool address' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus Name ResourceGroup ------------------------------------------------------------------------------ ----------------------------- ci-op-bv13g48p-4fcb7-s8dbc-rg_ci-op-bv13g48p-4fcb7-s8dbc-master-1-nicpipConfig ci-op-bv13g48p-4fcb7-s8dbc-rg ci-op-bv13g48p-4fcb7-s8dbc-rg_ci-op-bv13g48p-4fcb7-s8dbc-master-2-nicpipConfig ci-op-bv13g48p-4fcb7-s8dbc-rg ci-op-bv13g48p-4fcb7-s8dbc-rg_ci-op-bv13g48p-4fcb7-s8dbc-master-0-nicpipConfig ci-op-bv13g48p-4fcb7-s8dbc-rg
Expected results:
install succeed
Running Command: az network lb address-pool address list --lb-name ci-op-v0wp9w20-01c08-strqp --pool-name ci-op-v0wp9w20-01c08-strqp --resource-group ci-op-v0wp9w20-01c08-strqp-rg -o table WARNING: Command group 'network lb address-pool address' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus Name ResourceGroup ------------------------------------------------------------------------------ ----------------------------- ci-op-v0wp9w20-01c08-strqp-rg_ci-op-v0wp9w20-01c08-strqp-master-2-nicpipConfig ci-op-v0wp9w20-01c08-strqp-rg ci-op-v0wp9w20-01c08-strqp-rg_ci-op-v0wp9w20-01c08-strqp-master-1-nicpipConfig ci-op-v0wp9w20-01c08-strqp-rg ci-op-v0wp9w20-01c08-strqp-rg_ci-op-v0wp9w20-01c08-strqp-master-0-nicpipConfig ci-op-v0wp9w20-01c08-strqp-rg 3a04a295-a771-4cac-bca2-ed037a8c43ec ci-op-v0wp9w20-01c08-strqp-rg 177a23f8-bdf7-4bfd-b4f2-c3a447897d7f ci-op-v0wp9w20-01c08-strqp-rg c27d671e-0b20-4c03-9f66-5e8fd56f3883 ci-op-v0wp9w20-01c08-strqp-rg Running Command: az network lb address-pool address list --lb-name ci-op-v0wp9w20-01c08-strqp-internal --pool-name ci-op-v0wp9w20-01c08-strqp --resource-group ci-op-v0wp9w20-01c08-strqp-rg -o table WARNING: Command group 'network lb address-pool address' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus Name ResourceGroup ------------------------------------------------------------------------------ ----------------------------- ci-op-v0wp9w20-01c08-strqp-rg_ci-op-v0wp9w20-01c08-strqp-master-1-nicpipConfig ci-op-v0wp9w20-01c08-strqp-rg ci-op-v0wp9w20-01c08-strqp-rg_ci-op-v0wp9w20-01c08-strqp-master-0-nicpipConfig ci-op-v0wp9w20-01c08-strqp-rg ci-op-v0wp9w20-01c08-strqp-rg_ci-op-v0wp9w20-01c08-strqp-master-2-nicpipConfig ci-op-v0wp9w20-01c08-strqp-rg f60960b6-f3cd-48d5-8f9b-c7205ec8ec9c ci-op-v0wp9w20-01c08-strqp-rg c17c4572-ede0-482c-b47e-9ac6b47b5842 ci-op-v0wp9w20-01c08-strqp-rg 6320a816-6d78-4bfe-87a0-704802f0f35b ci-op-v0wp9w20-01c08-strqp-rg
Additional info: