scripts/runtests.sh OSSM Operator version is 2.4.0 Output dir: /go/src/maistra-test-tool/tests/result-20230802171548 Executing tests against SMCP version 2.4 WARNING: Using insecure TLS client config. Setting this option is not supported! Login successful. You have access to 68 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Resetting cluster by deleting namespaces used in the test suite Output dir: /go/src/maistra-test-tool/tests/result-20230802171548/v2.4 ====== Executing tests in group 'interop' against SMCP v2.4 test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestSMCPMultiple test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestSMCPMultiple (0.00s) PASS ok github.com/maistra/maistra-test-tool/pkg/tests/non-dependant 2.212s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestIstiodPodFailsAfterRestarts test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestIstiodPodFailsAfterRestarts (0.00s) === RUN TestDeployOnInfraNodes test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestDeployOnInfraNodes (0.00s) === RUN TestDiscoverySelectors test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestDiscoverySelectors (0.00s) === RUN TestInitContainerNotRemovedDuringInjection test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestInitContainerNotRemovedDuringInjection (0.00s) === RUN TestIOR test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestIOR (0.00s) === RUN TestOperatorCanUpdatePrometheusConfigMap test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestOperatorCanUpdatePrometheusConfigMap (0.00s) === RUN TestRateLimiting test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestRateLimiting (0.00s) === RUN TestSMCPAddons test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestSMCPAddons (0.00s) === RUN TestSMCPAnnotations test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestSMCPAnnotations (0.00s) === RUN TestMustGather test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestMustGather (0.00s) === RUN TestTLSVersionSMCP smcp_tls_ssl_test.go:25: This test checks if the SMCP updated the tls.minProtocolVersion to TLSv1_0, TLSv1_1, and tls.maxProtocolVersion to TLSv1_3. smcp_tls_ssl_test.go:34: smcp_tls_ssl_test.go:34: STEP 1: Apply default SMCP and SMMR manifests smcp_tls_ssl_test.go:34: Wait for condition condition=Ready on smcp istio-system/basic... smcp_tls_ssl_test.go:34: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 smcp_tls_ssl_test.go:34: Wait for condition condition=Ready on smcp istio-system/basic... smcp_tls_ssl_test.go:34: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 smcp_tls_ssl_test.go:34: Wait for condition condition=Ready on smcp istio-system/basic... smcp_tls_ssl_test.go:34: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 smcp_tls_ssl_test.go:34: Wait for condition condition=Ready on smcp istio-system/basic... smcp_tls_ssl_test.go:34: SUCCESS: Condition condition=Ready met by smcp istio-system/basic === RUN TestTLSVersionSMCP/minVersion_TLSv1_0 smcp_tls_ssl_test.go:37: smcp_tls_ssl_test.go:37: STEP 1: Update SMCP spec.security.controlPlane.tls.minProtocolVersion: TLSv1_0 smcp_tls_ssl_test.go:39: Wait for condition condition=Ready on smcp istio-system/basic... smcp_tls_ssl_test.go:39: SUCCESS: Condition condition=Ready met by smcp istio-system/basic subtest.go:23: subtest.go:28: Subtest completed in 10.17s (excluding cleanup) --- PASS: TestTLSVersionSMCP/minVersion_TLSv1_0 (10.17s) === RUN TestTLSVersionSMCP/minVersion_TLSv1_1 smcp_tls_ssl_test.go:43: smcp_tls_ssl_test.go:43: STEP 1: Check to see if the SMCP minProtocolVersion is TLSv1_1 smcp_tls_ssl_test.go:45: Wait for condition condition=Ready on smcp istio-system/basic... smcp_tls_ssl_test.go:45: SUCCESS: Condition condition=Ready met by smcp istio-system/basic subtest.go:23: subtest.go:28: Subtest completed in 10.31s (excluding cleanup) --- PASS: TestTLSVersionSMCP/minVersion_TLSv1_1 (10.31s) === RUN TestTLSVersionSMCP/maxVersion_TLSv1_3 smcp_tls_ssl_test.go:49: smcp_tls_ssl_test.go:49: STEP 1: Check to see if the SMCP maxProtocolVersion is TLSv1_3 smcp_tls_ssl_test.go:51: Wait for condition condition=Ready on smcp istio-system/basic... smcp_tls_ssl_test.go:51: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 smcp_tls_ssl_test.go:51: Wait for condition condition=Ready on smcp istio-system/basic... smcp_tls_ssl_test.go:51: SUCCESS: Condition condition=Ready met by smcp istio-system/basic subtest.go:23: subtest.go:28: Subtest completed in 10.55s (excluding cleanup) --- PASS: TestTLSVersionSMCP/maxVersion_TLSv1_3 (10.55s) test.go:77: test.go:82: Test completed in 70.02s (excluding cleanup) smcp_tls_ssl_test.go:26: smcp_tls_ssl_test.go:26: Performing cleanup smcp_tls_ssl_test.go:31: Wait for condition condition=Ready on smcp istio-system/basic... smcp_tls_ssl_test.go:31: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 smcp_tls_ssl_test.go:31: Wait for condition condition=Ready on smcp istio-system/basic... smcp_tls_ssl_test.go:31: SUCCESS: Condition condition=Ready met by smcp istio-system/basic smcp_tls_ssl_test.go:26: Cleanup completed in 10.51s --- PASS: TestTLSVersionSMCP (80.54s) === RUN TestSMMRAutoCreationAndDeletion test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestSMMRAutoCreationAndDeletion (0.00s) === RUN TestSmoke smoke_test.go:46: Smoke Test for SMCP: deploy, upgrade, bookinfo and uninstall smoke_test.go:56: Deleting namespaces: [istio-system] smoke_test.go:56: Creating namespaces: [istio-system] === RUN TestSmoke/upgrade_v2.3_to_v2.4 smoke_test.go:59: This test checks whether SMCP becomes ready after it's upgraded from v2.3 to v2.4 and bookinfo is still working after the upgrade and also test a clean installation of the target SMCP smoke_test.go:65: smoke_test.go:65: STEP 1: Install SMCP v2.3 and verify it becomes ready smoke_test.go:187: smoke_test.go:187: STEP 2: Install SMCP smoke_test.go:189: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:189: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 smoke_test.go:189: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:189: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 smoke_test.go:189: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:189: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 smoke_test.go:189: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:189: SUCCESS: Condition condition=Ready met by smcp istio-system/basic smoke_test.go:191: smoke_test.go:191: STEP 3: Check SMCP is Ready smoke_test.go:192: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:192: SUCCESS: Condition condition=Ready met by smcp istio-system/basic smoke_test.go:68: smoke_test.go:68: STEP 4: Install bookinfo pods and sleep pod smoke_test.go:69: Install app "bookinfo" in namespace "bookinfo" smoke_test.go:69: Create Bookinfo Gateway smoke_test.go:69: Create Bookinfo Destination Rules (all) smoke_test.go:69: Create Bookinfo Deployments smoke_test.go:69: Install app "sleep" in namespace "bookinfo" smoke_test.go:69: Wait for app bookinfo/bookinfo to be ready smoke_test.go:69: Wait for app bookinfo/sleep to be ready smoke_test.go:71: smoke_test.go:71: STEP 5: Check if bookinfo traffic flows through the Proxy smoke_test.go:138: SUCCESS: ProductPage returns 200 OK smoke_test.go:138: SUCCESS: HTTP header 'server: istio-envoy' is present in the response smoke_test.go:138: SUCCESS: HTTP header 'x-envoy-decorator-operation' is present in the response smoke_test.go:74: smoke_test.go:74: STEP 6: Upgrade SMCP from v2.3 to v2.4 smoke_test.go:187: smoke_test.go:187: STEP 7: Install SMCP smoke_test.go:189: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:189: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 smoke_test.go:189: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:189: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 smoke_test.go:189: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:189: SUCCESS: Condition condition=Ready met by smcp istio-system/basic smoke_test.go:191: smoke_test.go:191: STEP 8: Check SMCP is Ready smoke_test.go:192: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:192: SUCCESS: Condition condition=Ready met by smcp istio-system/basic smoke_test.go:77: smoke_test.go:77: STEP 9: Check if bookinfo productpage is running through the Proxy after the upgrade smoke_test.go:138: SUCCESS: ProductPage returns 200 OK smoke_test.go:138: SUCCESS: HTTP header 'server: istio-envoy' is present in the response smoke_test.go:138: SUCCESS: HTTP header 'x-envoy-decorator-operation' is present in the response smoke_test.go:80: smoke_test.go:80: STEP 10: Delete Bookinfo pods to force the update of the sidecar smoke_test.go:125: smoke_test.go:125: STEP 11: Verify if all the routes are created smoke_test.go:196: Related issue: https://issues.redhat.com/browse/OSSM-4069 smoke_test.go:198: SUCCESS: Route grafana is created smoke_test.go:198: SUCCESS: Route istio-ingressgateway is created smoke_test.go:198: SUCCESS: Route jaeger is created smoke_test.go:198: SUCCESS: Route kiali is created smoke_test.go:198: SUCCESS: Route prometheus is created smoke_test.go:128: smoke_test.go:128: STEP 12: Check if bookinfo traffic flows through the Proxy smoke_test.go:138: SUCCESS: ProductPage returns 200 OK smoke_test.go:138: SUCCESS: HTTP header 'server: istio-envoy' is present in the response smoke_test.go:138: SUCCESS: HTTP header 'x-envoy-decorator-operation' is present in the response smoke_test.go:131: smoke_test.go:131: STEP 13: verify proxy startup time. Expected to be less than 10 seconds smoke_test.go:132: Jira related: https://issues.redhat.com/browse/OSSM-3586 smoke_test.go:157: Extracting proxy startup time and last transition time for all the pods in the namespace subtest.go:23: subtest.go:28: Subtest completed in 113.28s (excluding cleanup) smoke_test.go:60: smoke_test.go:60: Performing cleanup smoke_test.go:61: Uninstalling Bookinfo from namespace "bookinfo" smoke_test.go:62: Deleting namespaces: [istio-system] smoke_test.go:62: Creating namespaces: [istio-system] smoke_test.go:60: Cleanup completed in 37.73s --- PASS: TestSmoke/upgrade_v2.3_to_v2.4 (151.01s) === RUN TestSmoke/install_smcp_v2.4 smoke_test.go:87: This test checks whether SMCP v2.4 install the SMCP version smoke_test.go:92: smoke_test.go:92: STEP 1: Install SMCP v2.4 smoke_test.go:187: smoke_test.go:187: STEP 2: Install SMCP smoke_test.go:189: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:189: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 smoke_test.go:189: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:189: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 smoke_test.go:189: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:189: SUCCESS: Condition condition=Ready met by smcp istio-system/basic smoke_test.go:191: smoke_test.go:191: STEP 3: Check SMCP is Ready smoke_test.go:192: Wait for condition condition=Ready on smcp istio-system/basic... smoke_test.go:192: SUCCESS: Condition condition=Ready met by smcp istio-system/basic smoke_test.go:95: smoke_test.go:95: STEP 4: Install bookinfo pods and sleep pod smoke_test.go:96: Install app "bookinfo" in namespace "bookinfo" smoke_test.go:96: Create Bookinfo Gateway smoke_test.go:96: Create Bookinfo Destination Rules (all) smoke_test.go:96: Create Bookinfo Deployments smoke_test.go:96: Install app "sleep" in namespace "bookinfo" smoke_test.go:96: Wait for app bookinfo/bookinfo to be ready smoke_test.go:96: Wait for app bookinfo/sleep to be ready smoke_test.go:125: smoke_test.go:125: STEP 5: Verify if all the routes are created smoke_test.go:196: Related issue: https://issues.redhat.com/browse/OSSM-4069 smoke_test.go:198: SUCCESS: Route grafana is created smoke_test.go:198: SUCCESS: Route istio-ingressgateway is created smoke_test.go:198: SUCCESS: Route jaeger is created smoke_test.go:198: FAILURE: Still waiting for route kiali to be created in namespace; expected to find the string 'kiali' in the output, but it wasn't found smoke_test.go:198: SUCCESS: Route prometheus is created smoke_test.go:197: --- Attempt 1/60 failed. Retrying... smoke_test.go:198: SUCCESS: Route grafana is created smoke_test.go:198: SUCCESS: Route istio-ingressgateway is created smoke_test.go:198: SUCCESS: Route jaeger is created smoke_test.go:198: FAILURE: Still waiting for route kiali to be created in namespace; expected to find the string 'kiali' in the output, but it wasn't found smoke_test.go:198: SUCCESS: Route prometheus is created smoke_test.go:197: --- Attempt 2/60 failed. Retrying... smoke_test.go:198: SUCCESS: Route grafana is created smoke_test.go:198: SUCCESS: Route istio-ingressgateway is created smoke_test.go:198: SUCCESS: Route jaeger is created smoke_test.go:198: FAILURE: Still waiting for route kiali to be created in namespace; expected to find the string 'kiali' in the output, but it wasn't found smoke_test.go:198: SUCCESS: Route prometheus is created smoke_test.go:197: --- Attempt 3/60 failed. Retrying... smoke_test.go:198: SUCCESS: Route grafana is created smoke_test.go:198: SUCCESS: Route istio-ingressgateway is created smoke_test.go:198: SUCCESS: Route jaeger is created smoke_test.go:198: FAILURE: Still waiting for route kiali to be created in namespace; expected to find the string 'kiali' in the output, but it wasn't found smoke_test.go:198: SUCCESS: Route prometheus is created smoke_test.go:197: --- Attempt 4/60 failed. Retrying... smoke_test.go:198: SUCCESS: Route grafana is created smoke_test.go:198: SUCCESS: Route istio-ingressgateway is created smoke_test.go:198: SUCCESS: Route jaeger is created smoke_test.go:198: FAILURE: Still waiting for route kiali to be created in namespace; expected to find the string 'kiali' in the output, but it wasn't found smoke_test.go:198: SUCCESS: Route prometheus is created smoke_test.go:197: --- Attempt 5/60 failed. Retrying... smoke_test.go:198: SUCCESS: Route grafana is created smoke_test.go:198: SUCCESS: Route istio-ingressgateway is created smoke_test.go:198: SUCCESS: Route jaeger is created smoke_test.go:198: FAILURE: Still waiting for route kiali to be created in namespace; expected to find the string 'kiali' in the output, but it wasn't found smoke_test.go:198: SUCCESS: Route prometheus is created smoke_test.go:197: --- Attempt 6/60 failed. Retrying... smoke_test.go:198: SUCCESS: Route grafana is created smoke_test.go:198: SUCCESS: Route istio-ingressgateway is created smoke_test.go:198: SUCCESS: Route jaeger is created smoke_test.go:198: FAILURE: Still waiting for route kiali to be created in namespace; expected to find the string 'kiali' in the output, but it wasn't found smoke_test.go:198: SUCCESS: Route prometheus is created smoke_test.go:197: --- Attempt 7/60 failed. Retrying... smoke_test.go:198: SUCCESS: Route grafana is created smoke_test.go:198: SUCCESS: Route istio-ingressgateway is created smoke_test.go:198: SUCCESS: Route jaeger is created smoke_test.go:198: FAILURE: Still waiting for route kiali to be created in namespace; expected to find the string 'kiali' in the output, but it wasn't found smoke_test.go:198: SUCCESS: Route prometheus is created smoke_test.go:197: --- Attempt 8/60 failed. Retrying... smoke_test.go:198: SUCCESS: Route grafana is created smoke_test.go:198: SUCCESS: Route istio-ingressgateway is created smoke_test.go:198: SUCCESS: Route jaeger is created smoke_test.go:198: FAILURE: Still waiting for route kiali to be created in namespace; expected to find the string 'kiali' in the output, but it wasn't found smoke_test.go:198: SUCCESS: Route prometheus is created smoke_test.go:197: --- Attempt 9/60 failed. Retrying... smoke_test.go:198: SUCCESS: Route grafana is created smoke_test.go:198: SUCCESS: Route istio-ingressgateway is created smoke_test.go:198: SUCCESS: Route jaeger is created smoke_test.go:198: FAILURE: Still waiting for route kiali to be created in namespace; expected to find the string 'kiali' in the output, but it wasn't found smoke_test.go:198: SUCCESS: Route prometheus is created smoke_test.go:197: --- Attempt 10/60 failed. Retrying... smoke_test.go:198: SUCCESS: Route grafana is created smoke_test.go:198: SUCCESS: Route istio-ingressgateway is created smoke_test.go:198: SUCCESS: Route jaeger is created smoke_test.go:198: SUCCESS: Route kiali is created smoke_test.go:198: SUCCESS: Route prometheus is created smoke_test.go:197: --- Attempt 11/60 successful; total time: 11.33s smoke_test.go:128: smoke_test.go:128: STEP 6: Check if bookinfo traffic flows through the Proxy smoke_test.go:138: SUCCESS: ProductPage returns 200 OK smoke_test.go:138: SUCCESS: HTTP header 'server: istio-envoy' is present in the response smoke_test.go:138: SUCCESS: HTTP header 'x-envoy-decorator-operation' is present in the response smoke_test.go:131: smoke_test.go:131: STEP 7: verify proxy startup time. Expected to be less than 10 seconds smoke_test.go:132: Jira related: https://issues.redhat.com/browse/OSSM-3586 smoke_test.go:157: Extracting proxy startup time and last transition time for all the pods in the namespace subtest.go:23: subtest.go:28: Subtest completed in 48.11s (excluding cleanup) smoke_test.go:88: smoke_test.go:88: Performing cleanup smoke_test.go:89: Uninstalling Bookinfo from namespace "bookinfo" smoke_test.go:88: Cleanup completed in 1.97s --- PASS: TestSmoke/install_smcp_v2.4 (50.08s) === RUN TestSmoke/delete_smcp_v2.4 smoke_test.go:102: This test checks whether SMCP v2.4 deletion deletes all the resources smoke_test.go:107: smoke_test.go:107: STEP 1: Delete SMCP and SMMR in namespace istio-system smoke_test.go:110: smoke_test.go:110: STEP 2: verify SMCP resources are deleted smoke_test.go:112: FAILURE: Still waiting for resources to be deleted from namespace; expected to find the string 'No resources found in' in the output, but it wasn't found smoke_test.go:111: --- Attempt 1/60 failed. Retrying... smoke_test.go:112: FAILURE: Still waiting for resources to be deleted from namespace; expected to find the string 'No resources found in' in the output, but it wasn't found smoke_test.go:111: --- Attempt 2/60 failed. Retrying... smoke_test.go:112: FAILURE: Still waiting for resources to be deleted from namespace; expected to find the string 'No resources found in' in the output, but it wasn't found smoke_test.go:111: --- Attempt 3/60 failed. Retrying... smoke_test.go:112: FAILURE: Still waiting for resources to be deleted from namespace; expected to find the string 'No resources found in' in the output, but it wasn't found smoke_test.go:111: --- Attempt 4/60 failed. Retrying... smoke_test.go:112: FAILURE: Still waiting for resources to be deleted from namespace; expected to find the string 'No resources found in' in the output, but it wasn't found smoke_test.go:111: --- Attempt 5/60 failed. Retrying... smoke_test.go:112: FAILURE: Still waiting for resources to be deleted from namespace; expected to find the string 'No resources found in' in the output, but it wasn't found smoke_test.go:111: --- Attempt 6/60 failed. Retrying... smoke_test.go:112: FAILURE: Still waiting for resources to be deleted from namespace; expected to find the string 'No resources found in' in the output, but it wasn't found smoke_test.go:111: --- Attempt 7/60 failed. Retrying... smoke_test.go:112: FAILURE: Still waiting for resources to be deleted from namespace; expected to find the string 'No resources found in' in the output, but it wasn't found smoke_test.go:111: --- Attempt 8/60 failed. Retrying... smoke_test.go:112: FAILURE: Still waiting for resources to be deleted from namespace; expected to find the string 'No resources found in' in the output, but it wasn't found smoke_test.go:111: --- Attempt 9/60 failed. Retrying... smoke_test.go:112: FAILURE: Still waiting for resources to be deleted from namespace; expected to find the string 'No resources found in' in the output, but it wasn't found smoke_test.go:111: --- Attempt 10/60 failed. Retrying... smoke_test.go:112: FAILURE: Still waiting for resources to be deleted from namespace; expected to find the string 'No resources found in' in the output, but it wasn't found smoke_test.go:111: --- Attempt 11/60 failed. Retrying... smoke_test.go:112: SUCCESS: SMCP resources are deleted smoke_test.go:111: --- Attempt 12/60 successful; total time: 12.79s subtest.go:23: subtest.go:28: Subtest completed in 14.84s (excluding cleanup) smoke_test.go:103: smoke_test.go:103: Performing cleanup smoke_test.go:104: Deleting namespaces: [istio-system] smoke_test.go:104: Creating namespaces: [istio-system] smoke_test.go:103: Cleanup completed in 20.71s --- PASS: TestSmoke/delete_smcp_v2.4 (35.55s) test.go:77: test.go:82: Test completed in 318.12s (excluding cleanup) smoke_test.go:49: smoke_test.go:49: Performing cleanup smoke_test.go:50: Deleting namespaces: [istio-system] smoke_test.go:50: Creating namespaces: [istio-system] smoke_test.go:49: Cleanup completed in 5.51s --- PASS: TestSmoke (323.63s) === RUN TestSSL testssl_test.go:45: testssl_test.go:45: STEP 1: Apply default SMCP and SMMR manifests testssl_test.go:45: Wait for condition condition=Ready on smcp istio-system/basic... testssl_test.go:45: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 testssl_test.go:45: Wait for condition condition=Ready on smcp istio-system/basic... testssl_test.go:45: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 testssl_test.go:45: Wait for condition condition=Ready on smcp istio-system/basic... testssl_test.go:45: SUCCESS: Condition condition=Ready met by smcp istio-system/basic testssl_test.go:47: testssl_test.go:47: STEP 2: Patch SMCP to enable mTLS in dataPlane and controlPlane and set min/maxProtocolVersion, cipherSuites, and ecdhCurves testssl_test.go:64: Wait for condition condition=Ready on smcp istio-system/basic... testssl_test.go:64: SUCCESS: Condition condition=Ready met by smcp istio-system/basic testssl_test.go:66: testssl_test.go:66: STEP 3: Install bookinfo with mTLS and testssl pod testssl_test.go:68: Install app "bookinfo" in namespace "bookinfo" testssl_test.go:68: Create Bookinfo Gateway testssl_test.go:68: Create Bookinfo Destination Rules (all) testssl_test.go:68: Create Bookinfo Deployments testssl_test.go:68: Wait for app bookinfo/bookinfo to be ready testssl_test.go:71: testssl_test.go:71: STEP 4: Check testssl.sh results testssl_test.go:73: SUCCESS: Received the TLSv1.2 needed in the testssl.sh results testssl_test.go:73: SUCCESS: Results received the correct SHA256 testssl_test.go:73: SUCCESS: Results included: P-256 test.go:77: test.go:82: Test completed in 50.49s (excluding cleanup) testssl_test.go:31: testssl_test.go:31: Performing cleanup testssl_test.go:41: Uninstalling Bookinfo from namespace "bookinfo" testssl_test.go:31: Cleanup completed in 2.47s --- PASS: TestSSL (52.96s) PASS ok github.com/maistra/maistra-test-tool/pkg/tests/ossm 458.537s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestClusterWideMode test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestClusterWideMode (0.00s) === RUN TestOperatorCanReconcileSMCPWhenIstiodOffline test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestOperatorCanReconcileSMCPWhenIstiodOffline (0.00s) === RUN TestOperatorPodHonorsReadinessProbe test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestOperatorPodHonorsReadinessProbe (0.00s) PASS ok github.com/maistra/maistra-test-tool/pkg/tests/ossm/operator 0.421s === RUN TestMultiClusterFederationFailover test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestMultiClusterFederationFailover (0.00s) === RUN TestFederation test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestFederation (0.00s) === RUN TestFederationDifferentCerts test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestFederationDifferentCerts (0.00s) PASS ok github.com/maistra/maistra-test-tool/pkg/tests/ossm-federation 0.003s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestCustomPrometheus test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestCustomPrometheus (0.00s) === RUN TestOpenShiftMonitoring test.go:112: This test is being skipped because it is not part of the "interop" test group --- SKIP: TestOpenShiftMonitoring (0.00s) PASS ok github.com/maistra/maistra-test-tool/pkg/tests/tasks/observability 0.424s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestAuthPolicy auth_test.go:45: auth_test.go:45: STEP 1: Apply default SMCP and SMMR manifests auth_test.go:45: Wait for condition condition=Ready on smcp istio-system/basic... auth_test.go:45: SUCCESS: Condition condition=Ready met by smcp istio-system/basic auth_test.go:47: auth_test.go:47: STEP 2: Install httpbin and sleep in multiple namespaces auth_test.go:48: Install app "httpbin" in namespace "foo" auth_test.go:48: Install app "httpbin" in namespace "bar" auth_test.go:48: Install app "httpbin" in namespace "legacy" auth_test.go:48: Install app "sleep" in namespace "foo" auth_test.go:48: Install app "sleep" in namespace "bar" auth_test.go:48: Install app "sleep" in namespace "legacy" auth_test.go:48: Wait for app foo/httpbin to be ready auth_test.go:48: Wait for app bar/httpbin to be ready auth_test.go:48: Wait for app legacy/httpbin to be ready auth_test.go:48: Wait for app foo/sleep to be ready auth_test.go:48: Wait for app bar/sleep to be ready auth_test.go:48: Wait for app legacy/sleep to be ready auth_test.go:59: auth_test.go:59: STEP 3: Check connectivity from namespaces foo, bar, and legacy to namespaces foo and bar mtls_migration_test.go:118: SUCCESS: foo connects to foo mtls_migration_test.go:118: SUCCESS: foo connects to bar mtls_migration_test.go:118: SUCCESS: bar connects to foo mtls_migration_test.go:118: SUCCESS: bar connects to bar mtls_migration_test.go:118: SUCCESS: legacy connects to foo mtls_migration_test.go:118: SUCCESS: legacy connects to bar === RUN TestAuthPolicy/enable_auto_mTLS auth_test.go:69: auth_test.go:69: STEP 1: Check if mTLS is enabled in foo auth_test.go:71: SUCCESS: mTLS is enabled in namespace foo (X-Forwarded-Client-Cert header is present) auth_test.go:80: auth_test.go:80: STEP 2: Check that mTLS is NOT enabled in legacy auth_test.go:82: SUCCESS: mTLS is not enabled in namespace legacy (X-Forwarded-Client-Cert header is not present) subtest.go:23: subtest.go:28: Subtest completed in 0.82s (excluding cleanup) --- PASS: TestAuthPolicy/enable_auto_mTLS (0.82s) === RUN TestAuthPolicy/enable_global_mTLS_STRICT_mode auth_test.go:93: auth_test.go:93: STEP 1: Enable mTLS STRICT mode globally auth_test.go:102: SUCCESS: Global mTLS expected 000 auth_test.go:102: SUCCESS: Global mTLS expected 000 subtest.go:23: subtest.go:28: Subtest completed in 1.08s (excluding cleanup) auth_test.go:95: auth_test.go:95: Performing cleanup auth_test.go:95: Cleanup completed in 0.13s --- PASS: TestAuthPolicy/enable_global_mTLS_STRICT_mode (1.21s) === RUN TestAuthPolicy/namespace_policy_mtls auth_test.go:114: auth_test.go:114: STEP 1: Enable mutual TLS per namespace mtls_migration_test.go:118: SUCCESS: foo connects to foo mtls_migration_test.go:118: SUCCESS: foo connects to bar mtls_migration_test.go:118: SUCCESS: bar connects to foo mtls_migration_test.go:118: SUCCESS: bar connects to bar mtls_migration_test.go:118: SUCCESS: legacy can't conect to foo mtls_migration_test.go:118: SUCCESS: legacy connects to bar subtest.go:23: subtest.go:28: Subtest completed in 2.77s (excluding cleanup) auth_test.go:116: auth_test.go:116: Performing cleanup auth_test.go:116: Cleanup completed in 0.12s --- PASS: TestAuthPolicy/namespace_policy_mtls (2.88s) === RUN TestAuthPolicy/workload_policy_mtls auth_test.go:133: auth_test.go:133: STEP 1: Enable mutual TLS per workload mtls_migration_test.go:118: SUCCESS: legacy can't conect to bar auth_test.go:142: auth_test.go:142: STEP 2: Refine mutual TLS per port mtls_migration_test.go:118: SUCCESS: legacy connects to bar subtest.go:23: subtest.go:28: Subtest completed in 1.65s (excluding cleanup) auth_test.go:135: auth_test.go:135: Performing cleanup auth_test.go:135: Cleanup completed in 0.16s --- PASS: TestAuthPolicy/workload_policy_mtls (1.80s) === RUN TestAuthPolicy/policy_precedence_mtls auth_test.go:150: auth_test.go:150: STEP 1: Overwrite foo namespace policy by a workload policy mtls_migration_test.go:118: SUCCESS: legacy connects to foo subtest.go:23: subtest.go:28: Subtest completed in 0.76s (excluding cleanup) auth_test.go:152: auth_test.go:152: Performing cleanup auth_test.go:152: Cleanup completed in 0.16s --- PASS: TestAuthPolicy/policy_precedence_mtls (0.92s) === RUN TestAuthPolicy/end-user_JWT auth_test.go:164: End-user authentication auth_test.go:166: auth_test.go:166: STEP 1: Apply httpbin gateway auth_test.go:169: auth_test.go:169: STEP 2: Check httpbin request is successful auth_test.go:230: SUCCESS: received expected status code 200 auth_test.go:174: auth_test.go:174: STEP 3: Apply a JWT policy auth_test.go:180: auth_test.go:180: STEP 4: Check whether request without token returns 200 auth_test.go:230: SUCCESS: received expected status code 200 auth_test.go:185: auth_test.go:185: STEP 5: Check whether request with an invalid token returns 401 auth_test.go:230: FAILURE: expected status code 401 but got 200 OK auth_test.go:186: --- Attempt 1/60 failed. Retrying... auth_test.go:230: FAILURE: expected status code 401 but got 200 OK auth_test.go:186: --- Attempt 2/60 failed. Retrying... auth_test.go:230: SUCCESS: received expected status code 401 auth_test.go:186: --- Attempt 3/60 successful; total time: 2.05s auth_test.go:190: auth_test.go:190: STEP 6: Check whether request with a valid token returns 200 auth_test.go:230: SUCCESS: received expected status code 200 subtest.go:23: subtest.go:28: Subtest completed in 4.06s (excluding cleanup) auth_test.go:176: auth_test.go:176: Performing cleanup auth_test.go:176: Cleanup completed in 0.13s --- PASS: TestAuthPolicy/end-user_JWT (4.19s) === RUN TestAuthPolicy/end-user_require_JWT auth_test.go:201: Require a valid token auth_test.go:230: FAILURE: expected status code 403 but got 200 OK auth_test.go:207: --- Attempt 1/60 failed. Retrying... auth_test.go:230: SUCCESS: received expected status code 403 auth_test.go:207: --- Attempt 2/60 successful; total time: 1.03s subtest.go:23: subtest.go:28: Subtest completed in 1.42s (excluding cleanup) auth_test.go:203: auth_test.go:203: Performing cleanup auth_test.go:203: Cleanup completed in 0.14s --- PASS: TestAuthPolicy/end-user_require_JWT (1.56s) === RUN TestAuthPolicy/end-user_require_JWT_per_path auth_test.go:213: Require valid tokens per-path auth_test.go:230: SUCCESS: received expected status code 403 auth_test.go:230: SUCCESS: received expected status code 200 subtest.go:23: subtest.go:28: Subtest completed in 0.51s (excluding cleanup) auth_test.go:215: auth_test.go:215: Performing cleanup auth_test.go:215: Cleanup completed in 0.12s --- PASS: TestAuthPolicy/end-user_require_JWT_per_path (0.63s) test.go:77: test.go:82: Test completed in 37.24s (excluding cleanup) auth_test.go:41: auth_test.go:41: Performing cleanup auth_test.go:42: Deleting namespaces: [foo bar legacy] auth_test.go:42: Creating namespaces: [foo bar legacy] auth_test.go:41: Cleanup completed in 18.63s --- PASS: TestAuthPolicy (55.86s) === RUN TestMTlsMigration mtls_migration_test.go:40: mtls_migration_test.go:40: STEP 1: Apply default SMCP and SMMR manifests mtls_migration_test.go:40: Wait for condition condition=Ready on smcp istio-system/basic... mtls_migration_test.go:40: SUCCESS: Condition condition=Ready met by smcp istio-system/basic mtls_migration_test.go:42: mtls_migration_test.go:42: STEP 2: Install httpbin and sleep in multiple namespaces mtls_migration_test.go:43: Install app "httpbin" in namespace "foo" mtls_migration_test.go:43: Install app "httpbin" in namespace "bar" mtls_migration_test.go:43: Install app "sleep" in namespace "foo" mtls_migration_test.go:43: Install app "sleep" in namespace "bar" mtls_migration_test.go:43: Install app "sleep" in namespace "legacy" mtls_migration_test.go:43: Wait for app foo/httpbin to be ready mtls_migration_test.go:43: Wait for app bar/httpbin to be ready mtls_migration_test.go:43: Wait for app foo/sleep to be ready mtls_migration_test.go:43: Wait for app bar/sleep to be ready mtls_migration_test.go:43: Wait for app legacy/sleep to be ready mtls_migration_test.go:53: mtls_migration_test.go:53: STEP 3: Check connectivity from namespaces foo, bar, and legacy to namespace foo and bar mtls_migration_test.go:118: SUCCESS: foo connects to foo mtls_migration_test.go:118: SUCCESS: foo connects to bar mtls_migration_test.go:118: SUCCESS: bar connects to foo mtls_migration_test.go:118: SUCCESS: bar connects to bar mtls_migration_test.go:118: SUCCESS: legacy connects to foo mtls_migration_test.go:118: SUCCESS: legacy connects to bar === RUN TestMTlsMigration/mTLS_enabled_in_foo mtls_migration_test.go:63: mtls_migration_test.go:63: STEP 1: Apply strict mTLS in namespace foo mtls_migration_test.go:66: mtls_migration_test.go:66: STEP 2: Check connectivity from namespaces foo, bar, and legacy to namespace foo and bar (expect failure only from legacy to foo) mtls_migration_test.go:118: SUCCESS: foo connects to foo mtls_migration_test.go:118: SUCCESS: foo connects to bar mtls_migration_test.go:118: SUCCESS: bar connects to foo mtls_migration_test.go:118: SUCCESS: bar connects to bar mtls_migration_test.go:118: SUCCESS: legacy can't conect to foo mtls_migration_test.go:118: SUCCESS: legacy connects to bar subtest.go:23: subtest.go:28: Subtest completed in 2.86s (excluding cleanup) --- PASS: TestMTlsMigration/mTLS_enabled_in_foo (2.86s) === RUN TestMTlsMigration/mTLS_enabled_globally mtls_migration_test.go:81: mtls_migration_test.go:81: STEP 1: Apply strict mTLS for the entire mesh mtls_migration_test.go:87: mtls_migration_test.go:87: STEP 2: Check connectivity from namespaces foo, bar, and legacy to namespace foo and bar (expect failure from legacy) mtls_migration_test.go:118: SUCCESS: foo connects to foo mtls_migration_test.go:118: SUCCESS: foo connects to bar mtls_migration_test.go:118: SUCCESS: bar connects to foo mtls_migration_test.go:118: SUCCESS: bar connects to bar mtls_migration_test.go:118: SUCCESS: legacy can't conect to foo mtls_migration_test.go:118: SUCCESS: legacy can't conect to bar subtest.go:23: subtest.go:28: Subtest completed in 2.87s (excluding cleanup) mtls_migration_test.go:83: mtls_migration_test.go:83: Performing cleanup mtls_migration_test.go:83: Cleanup completed in 0.12s --- PASS: TestMTlsMigration/mTLS_enabled_globally (2.99s) test.go:77: test.go:82: Test completed in 21.41s (excluding cleanup) mtls_migration_test.go:36: mtls_migration_test.go:36: Performing cleanup mtls_migration_test.go:37: Deleting namespaces: [foo bar legacy] mtls_migration_test.go:37: Creating namespaces: [foo bar legacy] mtls_migration_test.go:36: Cleanup completed in 18.55s --- PASS: TestMTlsMigration (39.97s) PASS ok github.com/maistra/maistra-test-tool/pkg/tests/tasks/security/authentication 96.235s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestAuthorizationDenyAllow deny_test.go:33: This test validates authorization policies with a deny action deny_test.go:35: deny_test.go:35: STEP 1: Apply default SMCP and SMMR manifests deny_test.go:35: Wait for condition condition=Ready on smcp istio-system/basic... deny_test.go:35: SUCCESS: Condition condition=Ready met by smcp istio-system/basic deny_test.go:37: deny_test.go:37: STEP 2: Install httpbin and sleep deny_test.go:38: Install app "httpbin" in namespace "foo" deny_test.go:38: Install app "sleep" in namespace "foo" deny_test.go:38: Wait for app foo/httpbin to be ready deny_test.go:38: Wait for app foo/sleep to be ready deny_test.go:40: deny_test.go:40: STEP 3: Check if httpbin returns 200 OK when no authorization policies are in place deny_test.go:41: SUCCESS: Got expected 200 OK from httpbin === RUN TestAuthorizationDenyAllow/explicitly_deny_request deny_test.go:47: deny_test.go:47: STEP 1: Apply policy that denies all GET requests to httpbin deny_test.go:50: deny_test.go:50: STEP 2: Verify that GET request is denied deny_test.go:51: SUCCESS: Got the expected 403 response code deny_test.go:53: deny_test.go:53: STEP 3: Verify that POST request is allowed deny_test.go:54: SUCCESS: Got the expected 200 OK response for request from httpbin subtest.go:23: subtest.go:28: Subtest completed in 1.21s (excluding cleanup) deny_test.go:44: deny_test.go:44: Performing cleanup deny_test.go:44: Cleanup completed in 0.11s --- PASS: TestAuthorizationDenyAllow/explicitly_deny_request (1.32s) === RUN TestAuthorizationDenyAllow/deny_request_header deny_test.go:61: deny_test.go:61: STEP 1: Apply policy that denies GET requests unless the HTTP header 'x-token: admin' is present deny_test.go:64: deny_test.go:64: STEP 2: Verify that GET request with HTTP header 'x-token: admin' is allowed deny_test.go:65: FAILURE: Expected the AuthorizationPolicy to accept request (expected HTTP status 200), but got a different HTTP code; expected to find the string '200' in the output, but it wasn't found deny_test.go:65: --- Attempt 1/60 failed. Retrying... deny_test.go:65: SUCCESS: Got the expected 200 OK response for request from httpbin deny_test.go:65: --- Attempt 2/60 successful; total time: 1.82s deny_test.go:67: deny_test.go:67: STEP 3: Verify that GET request with HTTP header 'x-token: guest' is denied deny_test.go:68: SUCCESS: Got the expected 403 response code subtest.go:23: subtest.go:28: Subtest completed in 2.63s (excluding cleanup) deny_test.go:58: deny_test.go:58: Performing cleanup deny_test.go:58: Cleanup completed in 0.12s --- PASS: TestAuthorizationDenyAllow/deny_request_header (2.75s) === RUN TestAuthorizationDenyAllow/allow_request_path deny_test.go:76: deny_test.go:76: STEP 1: Apply policy that denies GET requests unless the HTTP header 'x-token: admin' is present deny_test.go:79: deny_test.go:79: STEP 2: Apply policy that allows requests with the path '/ip' deny_test.go:82: deny_test.go:82: STEP 3: Verify that GET request with the HTTP header 'x-token: guest' at path '/ip' is denied deny_test.go:83: SUCCESS: Got the expected 403 response code deny_test.go:85: deny_test.go:85: STEP 4: Verify that GET request with HTTP header 'x-token: admin' at path '/ip' is allowed deny_test.go:86: SUCCESS: Got the expected 200 OK response for request from httpbin deny_test.go:88: deny_test.go:88: STEP 5: Verify that GET request with HTTP header 'x-token: admin' at path '/get' is denied deny_test.go:89: SUCCESS: Got the expected 403 response code subtest.go:23: subtest.go:28: Subtest completed in 1.92s (excluding cleanup) deny_test.go:72: deny_test.go:72: Performing cleanup deny_test.go:72: Cleanup completed in 0.26s --- PASS: TestAuthorizationDenyAllow/allow_request_path (2.18s) test.go:77: test.go:82: Test completed in 18.31s (excluding cleanup) deny_test.go:29: deny_test.go:29: Performing cleanup deny_test.go:30: Deleting namespaces: [foo] deny_test.go:30: Creating namespaces: [foo] deny_test.go:29: Cleanup completed in 17.51s --- PASS: TestAuthorizationDenyAllow (35.82s) === RUN TestEnvoyExtAuthzHttpExtensionProvider ext_auth_test.go:33: This test validates authorization policies with a JWT Token ext_auth_test.go:37: ext_auth_test.go:37: STEP 1: Apply default SMCP and SMMR manifests ext_auth_test.go:37: Wait for condition condition=Ready on smcp istio-system/basic... ext_auth_test.go:37: SUCCESS: Condition condition=Ready met by smcp istio-system/basic ext_auth_test.go:39: ext_auth_test.go:39: STEP 2: Install httpbin and sleep ext_auth_test.go:40: Install app "httpbin" in namespace "foo" ext_auth_test.go:40: Install app "sleep" in namespace "foo" ext_auth_test.go:40: Wait for app foo/httpbin to be ready ext_auth_test.go:40: Wait for app foo/sleep to be ready ext_auth_test.go:45: ext_auth_test.go:45: STEP 3: Check if httpbin returns 200 OK when no authorization policies are in place ext_auth_test.go:46: SUCCESS: Got expected 200 OK from httpbin ext_auth_test.go:48: ext_auth_test.go:48: STEP 4: Deploy the External Authorizer and Verify the sample external authorizer is up and running ext_auth_test.go:56: ext_auth_test.go:56: STEP 5: Set envoyExtAuthzHttp extension provider in SMCP ext_auth_test.go:93: ext_auth_test.go:93: STEP 6: Deploy the external authorization in the Authorization policy ext_auth_test.go:99: ext_auth_test.go:99: STEP 7: Verify a request to path /headers with header x-ext-authz: deny is denied by the sample ext_authz server: ext_auth_test.go:100: SUCCESS: Got the expected 403 response code ext_auth_test.go:102: ext_auth_test.go:102: STEP 8: Verify a request to path /headers with header x-ext-authz: allow is allowed by the sample ext_authz server ext_auth_test.go:103: SUCCESS: Got the expected 200 OK response for request from httpbin ext_auth_test.go:105: ext_auth_test.go:105: STEP 9: Verify a request to path /ip is allowed and does not trigger the external authorization ext_auth_test.go:106: SUCCESS: Got expected 200 OK from httpbin test.go:77: test.go:82: Test completed in 20.39s (excluding cleanup) ext_auth_test.go:94: ext_auth_test.go:94: Performing cleanup ext_auth_test.go:94: Cleanup completed in 0.11s ext_auth_test.go:87: ext_auth_test.go:87: Performing cleanup ext_auth_test.go:87: Cleanup completed in 0.15s ext_auth_test.go:50: ext_auth_test.go:50: Performing cleanup ext_auth_test.go:50: Cleanup completed in 0.16s ext_auth_test.go:41: ext_auth_test.go:41: Performing cleanup ext_auth_test.go:41: Cleanup completed in 0.51s --- PASS: TestEnvoyExtAuthzHttpExtensionProvider (21.33s) === RUN TestEnvoyExtAuthzGrpcExtensionProvider ext_auth_test.go:115: This test validates authorization policies with a JWT Token ext_auth_test.go:119: ext_auth_test.go:119: STEP 1: Apply default SMCP and SMMR manifests ext_auth_test.go:119: Wait for condition condition=Ready on smcp istio-system/basic... ext_auth_test.go:119: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 ext_auth_test.go:119: Wait for condition condition=Ready on smcp istio-system/basic... ext_auth_test.go:119: SUCCESS: Condition condition=Ready met by smcp istio-system/basic ext_auth_test.go:121: ext_auth_test.go:121: STEP 2: Install httpbin and sleep ext_auth_test.go:122: Install app "httpbin" in namespace "foo" ext_auth_test.go:122: Install app "sleep" in namespace "foo" ext_auth_test.go:122: Wait for app foo/httpbin to be ready ext_auth_test.go:122: Wait for app foo/sleep to be ready ext_auth_test.go:127: ext_auth_test.go:127: STEP 3: Check if httpbin returns 200 OK when no authorization policies are in place ext_auth_test.go:128: SUCCESS: Got expected 200 OK from httpbin ext_auth_test.go:130: ext_auth_test.go:130: STEP 4: Deploy the External Authorizer and Verify the sample external authorizer is up and running ext_auth_test.go:138: ext_auth_test.go:138: STEP 5: Set envoyExtAuthzgRPC extension provider in SMCP ext_auth_test.go:159: ext_auth_test.go:159: STEP 6: Deploy the external authorization in the Authorization policy ext_auth_test.go:165: ext_auth_test.go:165: STEP 7: Verify a request to path /headers with header x-ext-authz: deny is denied by the sample ext_authz server: ext_auth_test.go:166: FAILURE: Expected the AuthorizationPolicy to reject request (expected HTTP status 403), but got a different HTTP code; expected to find the string '403' in the output, but it wasn't found ext_auth_test.go:166: --- Attempt 1/60 failed. Retrying... ext_auth_test.go:166: SUCCESS: Got the expected 403 response code ext_auth_test.go:166: --- Attempt 2/60 successful; total time: 1.88s ext_auth_test.go:168: ext_auth_test.go:168: STEP 8: Verify a request to path /headers with header x-ext-authz: allow is allowed by the sample ext_authz server ext_auth_test.go:169: SUCCESS: Got the expected 200 OK response for request from httpbin ext_auth_test.go:171: ext_auth_test.go:171: STEP 9: Verify a request to path /ip is allowed and does not trigger the external authorization ext_auth_test.go:172: SUCCESS: Got expected 200 OK from httpbin test.go:77: test.go:82: Test completed in 24.80s (excluding cleanup) ext_auth_test.go:160: ext_auth_test.go:160: Performing cleanup ext_auth_test.go:160: Cleanup completed in 0.12s ext_auth_test.go:152: ext_auth_test.go:152: Performing cleanup ext_auth_test.go:152: Cleanup completed in 0.15s ext_auth_test.go:132: ext_auth_test.go:132: Performing cleanup ext_auth_test.go:132: Cleanup completed in 0.28s ext_auth_test.go:123: ext_auth_test.go:123: Performing cleanup ext_auth_test.go:123: Cleanup completed in 0.57s --- PASS: TestEnvoyExtAuthzGrpcExtensionProvider (25.93s) === RUN TestAuthorizationHTTPTraffic http_test.go:42: This test validates authorization policies for HTTP traffic. http_test.go:43: Doc reference: https://istio.io/v1.14/docs/tasks/security/authorization/authz-http/ http_test.go:45: http_test.go:45: STEP 1: Apply default SMCP and SMMR manifests http_test.go:45: Wait for condition condition=Ready on smcp istio-system/basic... http_test.go:45: SUCCESS: Condition condition=Ready met by smcp istio-system/basic http_test.go:47: http_test.go:47: STEP 2: Enable Service Mesh Control Plane mTLS http_test.go:53: http_test.go:53: STEP 3: Install bookinfo with mTLS http_test.go:54: Install app "bookinfo" in namespace "bookinfo" http_test.go:54: Create Bookinfo Gateway http_test.go:54: Create Bookinfo Destination Rules (all) http_test.go:54: Create Bookinfo Deployments http_test.go:54: Wait for app bookinfo/bookinfo to be ready http_test.go:55: Wait for condition condition=Ready on smcp istio-system/basic... http_test.go:55: SUCCESS: Condition condition=Ready met by smcp istio-system/basic === RUN TestAuthorizationHTTPTraffic/deny_all_http_traffic_to_bookinfo http_test.go:63: http_test.go:63: STEP 1: Apply policy that denies all HTTP requests to bookinfo workloads http_test.go:66: http_test.go:66: STEP 2: Verify that GET request is denied http_test.go:68: FAILURE: expected to find the string 'RBAC: access denied' in the response, but it wasn't found http_test.go:67: --- Attempt 1/60 failed. Retrying... http_test.go:68: SUCCESS: string 'RBAC: access denied' found in response http_test.go:67: --- Attempt 2/60 successful; total time: 1.27s subtest.go:23: subtest.go:28: Subtest completed in 1.66s (excluding cleanup) http_test.go:60: http_test.go:60: Performing cleanup http_test.go:60: Cleanup completed in 0.12s --- PASS: TestAuthorizationHTTPTraffic/deny_all_http_traffic_to_bookinfo (1.79s) === RUN TestAuthorizationHTTPTraffic/only_allow_HTTP_GET_request_to_the_productpage_workload http_test.go:81: http_test.go:81: STEP 1: Apply policies that allow access with GET method to the productpage workload and deny requests to other workloads http_test.go:85: http_test.go:85: STEP 2: Verify that GET request to the productpage is allowed and fetching other services is denied http_test.go:87: FAILURE: expected to find the string 'Error fetching product details' in the response, but it wasn't found http_test.go:87: FAILURE: expected to find the string 'Error fetching product reviews' in the response, but it wasn't found http_test.go:86: --- Attempt 1/60 failed. Retrying... http_test.go:87: FAILURE: expected to find the string 'Error fetching product details' in the response, but it wasn't found http_test.go:87: SUCCESS: string 'Error fetching product reviews' found in response http_test.go:86: --- Attempt 2/60 failed. Retrying... http_test.go:87: SUCCESS: string 'Error fetching product details' found in response http_test.go:87: SUCCESS: string 'Error fetching product reviews' found in response http_test.go:86: --- Attempt 3/60 successful; total time: 2.12s subtest.go:23: subtest.go:28: Subtest completed in 2.85s (excluding cleanup) http_test.go:77: http_test.go:77: Performing cleanup http_test.go:77: Cleanup completed in 0.25s --- PASS: TestAuthorizationHTTPTraffic/only_allow_HTTP_GET_request_to_the_productpage_workload (3.09s) === RUN TestAuthorizationHTTPTraffic/allow_HTTP_GET_requests_to_all_bookinfo_workloads http_test.go:104: http_test.go:104: STEP 1: Apply policies that allow HTTP GET requests to all bookinfo workloads http_test.go:111: http_test.go:111: STEP 2: Verify that GET requests are allowed to all bookinfo workloads http_test.go:113: SUCCESS: string 'RBAC: access denied' not found in response http_test.go:113: SUCCESS: string 'Error fetching product details' not found in response http_test.go:113: SUCCESS: string 'Error fetching product reviews' not found in response http_test.go:113: SUCCESS: string 'Ratings service currently unavailable' not found in response subtest.go:23: subtest.go:28: Subtest completed in 2.41s (excluding cleanup) http_test.go:97: http_test.go:97: Performing cleanup http_test.go:97: Cleanup completed in 0.59s --- PASS: TestAuthorizationHTTPTraffic/allow_HTTP_GET_requests_to_all_bookinfo_workloads (3.00s) test.go:77: test.go:82: Test completed in 31.74s (excluding cleanup) http_test.go:33: http_test.go:33: Performing cleanup http_test.go:38: Deleting namespaces: [bookinfo] http_test.go:38: Creating namespaces: [bookinfo] http_test.go:39: Wait for condition condition=Ready on smcp istio-system/basic... http_test.go:39: SUCCESS: Condition condition=Ready met by smcp istio-system/basic http_test.go:33: Cleanup completed in 14.72s --- PASS: TestAuthorizationHTTPTraffic (46.46s) === RUN TestAuthorizationJWT jwt_test.go:35: This test validates authorization policies with JWT Token jwt_test.go:37: jwt_test.go:37: STEP 1: Apply default SMCP and SMMR manifests jwt_test.go:37: Wait for condition condition=Ready on smcp istio-system/basic... jwt_test.go:37: SUCCESS: Condition condition=Ready met by smcp istio-system/basic jwt_test.go:39: jwt_test.go:39: STEP 2: Install httpbin and sleep jwt_test.go:40: Install app "httpbin" in namespace "foo" jwt_test.go:40: Install app "sleep" in namespace "foo" jwt_test.go:40: Wait for app foo/httpbin to be ready jwt_test.go:40: Wait for app foo/sleep to be ready jwt_test.go:42: jwt_test.go:42: STEP 3: Check if httpbin returns 200 OK when no authorization policies are in place jwt_test.go:43: SUCCESS: Got expected 200 OK from httpbin === RUN TestAuthorizationJWT/Allow_requests_with_valid_JWT_and_list-typed_claims jwt_test.go:57: jwt_test.go:57: STEP 1: Verify that a request with an invalid JWT is denied jwt_test.go:58: SUCCESS: Got the expected 401 response code jwt_test.go:60: jwt_test.go:60: STEP 2: Verify that a request without a JWT is allowed because there is no authorization policy jwt_test.go:61: SUCCESS: Got the expected 200 OK response for request from httpbin subtest.go:23: subtest.go:28: Subtest completed in 0.83s (excluding cleanup) --- PASS: TestAuthorizationJWT/Allow_requests_with_valid_JWT_and_list-typed_claims (0.83s) === RUN TestAuthorizationJWT/Security_authorization_allow_JWT_requestPrincipal jwt_test.go:69: jwt_test.go:69: STEP 1: Verify that a request with a valid JWT is allowed jwt_test.go:70: SUCCESS: Got the expected 200 OK response for request from httpbin jwt_test.go:72: jwt_test.go:72: STEP 2: Verify request without a JWT is denied jwt_test.go:73: FAILURE: Expected the AuthorizationPolicy to reject request (expected HTTP status 403), but got a different HTTP code; expected to find the string '403' in the output, but it wasn't found jwt_test.go:73: --- Attempt 1/60 failed. Retrying... jwt_test.go:73: SUCCESS: Got the expected 403 response code jwt_test.go:73: --- Attempt 2/60 successful; total time: 1.85s subtest.go:23: subtest.go:28: Subtest completed in 2.62s (excluding cleanup) jwt_test.go:65: jwt_test.go:65: Performing cleanup jwt_test.go:65: Cleanup completed in 0.12s --- PASS: TestAuthorizationJWT/Security_authorization_allow_JWT_requestPrincipal (2.74s) === RUN TestAuthorizationJWT/Security_authorization_allow_JWT_claims_group jwt_test.go:81: jwt_test.go:81: STEP 1: Verify that a request with the JWT that includes group1 in the groups claim is allowed jwt_test.go:82: SUCCESS: Got the expected 200 OK response for request from httpbin jwt_test.go:84: jwt_test.go:84: STEP 2: Verify that a request with a JWT, which does not have the groups claim is rejected jwt_test.go:85: FAILURE: Expected the AuthorizationPolicy to reject request (expected HTTP status 403), but got a different HTTP code; expected to find the string '403' in the output, but it wasn't found jwt_test.go:85: --- Attempt 1/60 failed. Retrying... jwt_test.go:85: SUCCESS: Got the expected 403 response code jwt_test.go:85: --- Attempt 2/60 successful; total time: 1.82s subtest.go:23: subtest.go:28: Subtest completed in 2.57s (excluding cleanup) jwt_test.go:77: jwt_test.go:77: Performing cleanup jwt_test.go:77: Cleanup completed in 0.11s --- PASS: TestAuthorizationJWT/Security_authorization_allow_JWT_claims_group (2.68s) test.go:77: test.go:82: Test completed in 19.59s (excluding cleanup) jwt_test.go:51: jwt_test.go:51: Performing cleanup jwt_test.go:51: Cleanup completed in 0.13s jwt_test.go:31: jwt_test.go:31: Performing cleanup jwt_test.go:32: Deleting namespaces: [foo] jwt_test.go:32: Creating namespaces: [foo] jwt_test.go:31: Cleanup completed in 18.08s --- PASS: TestAuthorizationJWT (37.81s) === RUN TestAuthorizationTCPTraffic tcp_test.go:38: This test validates authorization policies for TCP traffic. tcp_test.go:39: Doc reference: https://istio.io/v1.14/docs/tasks/security/authorization/authz-tcp/ tcp_test.go:41: tcp_test.go:41: STEP 1: Apply default SMCP and SMMR manifests tcp_test.go:41: Wait for condition condition=Ready on smcp istio-system/basic... tcp_test.go:41: SUCCESS: Condition condition=Ready met by smcp istio-system/basic tcp_test.go:43: tcp_test.go:43: STEP 2: Install sleep and echo tcp_test.go:44: Install app "sleep" in namespace "foo" tcp_test.go:44: Install app "echo" in namespace "foo" tcp_test.go:44: Wait for app foo/sleep to be ready tcp_test.go:44: Wait for app foo/echo to be ready tcp_test.go:46: tcp_test.go:46: STEP 3: Verify sleep to echo TCP connections tcp_test.go:94: SUCCESS: Got expected hello message on port 9000 tcp_test.go:94: SUCCESS: Got expected hello message on port 9001 === RUN TestAuthorizationTCPTraffic/TCP_invalid_policy tcp_test.go:56: tcp_test.go:56: STEP 1: Apply an invalid policy to allow requests to port 9000 and add an HTTP GET field tcp_test.go:106: SUCCESS: Got expected connection rejected on port 9000 tcp_test.go:106: SUCCESS: Got expected connection rejected on port 9001 subtest.go:23: subtest.go:28: Subtest completed in 1.20s (excluding cleanup) tcp_test.go:53: tcp_test.go:53: Performing cleanup tcp_test.go:53: Cleanup completed in 0.14s --- PASS: TestAuthorizationTCPTraffic/TCP_invalid_policy (1.33s) === RUN TestAuthorizationTCPTraffic/TCP_deny_policy tcp_test.go:69: tcp_test.go:69: STEP 1: Apply a policy to deny tcp requests to port 9000 tcp_test.go:106: SUCCESS: Got expected connection rejected on port 9000 tcp_test.go:94: SUCCESS: Got expected hello message on port 9001 subtest.go:23: subtest.go:28: Subtest completed in 1.29s (excluding cleanup) tcp_test.go:66: tcp_test.go:66: Performing cleanup tcp_test.go:66: Cleanup completed in 0.12s --- PASS: TestAuthorizationTCPTraffic/TCP_deny_policy (1.40s) === RUN TestAuthorizationTCPTraffic/TCP_ALLOW_policy tcp_test.go:82: tcp_test.go:82: STEP 1: Apply a policy to allow tcp requests to port 9000 and 9001 tcp_test.go:94: SUCCESS: Got expected hello message on port 9000 tcp_test.go:94: SUCCESS: Got expected hello message on port 9001 subtest.go:23: subtest.go:28: Subtest completed in 1.23s (excluding cleanup) tcp_test.go:79: tcp_test.go:79: Performing cleanup tcp_test.go:79: Cleanup completed in 0.13s --- PASS: TestAuthorizationTCPTraffic/TCP_ALLOW_policy (1.36s) test.go:77: test.go:82: Test completed in 17.52s (excluding cleanup) tcp_test.go:34: tcp_test.go:34: Performing cleanup tcp_test.go:35: Deleting namespaces: [foo] tcp_test.go:35: Creating namespaces: [foo] tcp_test.go:34: Cleanup completed in 13.55s --- PASS: TestAuthorizationTCPTraffic (31.07s) === RUN TestTrustDomainMigration trust_domain_test.go:49: trust_domain_test.go:49: STEP 1: Apply default SMCP and SMMR manifests trust_domain_test.go:49: Wait for condition condition=Ready on smcp istio-system/basic... trust_domain_test.go:49: SUCCESS: Condition condition=Ready met by smcp istio-system/basic trust_domain_test.go:96: Configure spec.security.trust.domain to "old-td" and alias "" trust_domain_test.go:112: Wait for condition condition=Ready on smcp istio-system/basic... trust_domain_test.go:112: SUCCESS: Condition condition=Ready met by smcp istio-system/basic trust_domain_test.go:53: Install app "httpbin" in namespace "foo" trust_domain_test.go:53: Install app "sleep" in namespace "foo" trust_domain_test.go:53: Install app "sleep" in namespace "bar" trust_domain_test.go:53: Wait for app foo/httpbin to be ready trust_domain_test.go:53: Wait for app foo/sleep to be ready trust_domain_test.go:53: Wait for app bar/sleep to be ready trust_domain_test.go:58: Apply deny all policy except sleep in bar namespace === RUN TestTrustDomainMigration/Case_1:_Verifying_policy_works trust_domain_test.go:85: Verifying curl output, expecting 403 trust_domain_test.go:87: SUCCESS: string '403' found in output trust_domain_test.go:85: Verifying curl output, expecting 200 trust_domain_test.go:87: SUCCESS: string '200' found in output subtest.go:23: subtest.go:28: Subtest completed in 0.89s (excluding cleanup) --- PASS: TestTrustDomainMigration/Case_1:_Verifying_policy_works (0.89s) === RUN TestTrustDomainMigration/Case_2:_Migrate_trust_domain_without_trust_domain_aliases trust_domain_test.go:96: Configure spec.security.trust.domain to "new-td" and alias "" trust_domain_test.go:112: Wait for condition condition=Ready on smcp istio-system/basic... trust_domain_test.go:112: SUCCESS: Condition condition=Ready met by smcp istio-system/basic trust_domain_test.go:85: Verifying curl output, expecting 403 trust_domain_test.go:87: SUCCESS: string '403' found in output trust_domain_test.go:85: Verifying curl output, expecting 403 trust_domain_test.go:87: SUCCESS: string '403' found in output subtest.go:23: subtest.go:28: Subtest completed in 18.61s (excluding cleanup) --- PASS: TestTrustDomainMigration/Case_2:_Migrate_trust_domain_without_trust_domain_aliases (18.61s) === RUN TestTrustDomainMigration/Case_3:_Migrate_trust_domain_with_trust_domain_aliases trust_domain_test.go:96: Configure spec.security.trust.domain to "new-td" and alias "old-td" trust_domain_test.go:112: Wait for condition condition=Ready on smcp istio-system/basic... trust_domain_test.go:112: SUCCESS: Condition condition=Ready met by smcp istio-system/basic trust_domain_test.go:85: Verifying curl output, expecting 403 trust_domain_test.go:87: SUCCESS: string '403' found in output trust_domain_test.go:85: Verifying curl output, expecting 200 trust_domain_test.go:87: SUCCESS: string '200' found in output subtest.go:23: subtest.go:28: Subtest completed in 24.61s (excluding cleanup) --- PASS: TestTrustDomainMigration/Case_3:_Migrate_trust_domain_with_trust_domain_aliases (24.61s) test.go:77: test.go:82: Test completed in 68.48s (excluding cleanup) trust_domain_test.go:39: trust_domain_test.go:39: Performing cleanup trust_domain_test.go:39: Cleanup completed in 1.17s --- PASS: TestTrustDomainMigration (69.64s) PASS ok github.com/maistra/maistra-test-tool/pkg/tests/tasks/security/authorization 268.464s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestExternalCertificate external_ca_test.go:48: external_ca_test.go:48: STEP 1: Uninstall existing SMCP external_ca_test.go:49: Deleting namespaces: [istio-system] external_ca_test.go:49: Creating namespaces: [istio-system] external_ca_test.go:51: external_ca_test.go:51: STEP 2: Create cacerts secret external_ca_test.go:60: external_ca_test.go:60: STEP 3: Apply SMCP to configure certificate authority to use cacerts secret external_ca_test.go:62: Wait for condition condition=Ready on smcp istio-system/basic... external_ca_test.go:62: SUCCESS: Condition condition=Ready met by smcp istio-system/basic external_ca_test.go:64: external_ca_test.go:64: STEP 4: Install bookinfo external_ca_test.go:65: Install app "bookinfo" in namespace "bookinfo" external_ca_test.go:65: Create Bookinfo Gateway external_ca_test.go:65: Create Bookinfo Destination Rules (all) external_ca_test.go:65: Create Bookinfo Deployments external_ca_test.go:65: Wait for app bookinfo/bookinfo to be ready external_ca_test.go:67: external_ca_test.go:67: STEP 5: Wait for response from productpage app external_ca_test.go:69: SUCCESS: received expected status code 200 external_ca_test.go:74: external_ca_test.go:74: STEP 6: Retrieve certificates from bookinfo details service external_ca_test.go:86: SUCCESS: The cert-chain certificates are present in the certificates sent by the tested service external_ca_test.go:90: SUCCESS: Certificate trust chain successfully verified test.go:77: test.go:82: Test completed in 91.61s (excluding cleanup) external_ca_test.go:37: external_ca_test.go:37: Performing cleanup external_ca_test.go:38: Recreate namespace bookinfo external_ca_test.go:39: Deleting namespaces: [bookinfo istio-system] external_ca_test.go:39: Creating namespaces: [bookinfo istio-system] external_ca_test.go:37: Cleanup completed in 13.16s --- PASS: TestExternalCertificate (104.77s) PASS ok github.com/maistra/maistra-test-tool/pkg/tests/tasks/security/certificate 105.484s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestIstioCsr install.go:32: cert-manager-operator is not installed, starting installation install.go:59: install.go:59: STEP 1: Create namespace for cert-manager-operator install.go:60: Creating namespaces: [cert-manager-operator] install.go:62: install.go:62: STEP 2: Install cert-manager-operator install.go:67: Waiting for cert-manager-operator to succeed install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 1/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 2/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 3/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 4/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 5/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 6/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 7/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 8/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 9/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 10/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 11/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 12/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 13/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 14/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 15/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 16/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 17/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 18/60 failed. Retrying... install.go:69: --- Attempt 19/60 successful; total time: 20.34s install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s error: timed out waiting for the condition on clusterserviceversions/cert-manager-operator.v1.11.1 error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: SUCCESS: Condition jsonpath='{.status.phase}'=Succeeded met by csv cert-manager-operator/cert-manager-operator.v1.11.1 install.go:76: Pod cert-manager-operator-controller-manager-c5c6bddcf-n2mm4 in namespace cert-manager-operator is ready! install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 1/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 2/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 3/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 4/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 5/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 6/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 7/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 8/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 9/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 10/60 failed. Retrying in 4s... install.go:77: Pod cert-manager-84b449cdff-fsv2w in namespace cert-manager is ready! install.go:77: --- Attempt 11/60 successful; total time: 41.79s install.go:41: install.go:41: STEP 3: Create root ca istio_csr_test.go:63: istio_csr_test.go:63: STEP 4: Uninstall existing SMCP istio_csr_test.go:64: Deleting namespaces: [istio-system] istio_csr_test.go:64: Creating namespaces: [istio-system] istio_csr_test.go:66: istio_csr_test.go:66: STEP 5: Create intermediate certificate for Istio istio_csr_test.go:69: istio_csr_test.go:69: STEP 6: Add jetstack repo to helm istio_csr_test.go:72: istio_csr_test.go:72: STEP 7: Install cert-manager-istio-csr istio_csr_test.go:81: istio_csr_test.go:81: STEP 8: Deploy SMCP v2.4 and SMMR istio_csr_test.go:83: Wait for condition condition=Ready on smcp istio-system/basic... istio_csr_test.go:83: SUCCESS: Condition condition=Ready met by smcp istio-system/basic istio_csr_test.go:85: istio_csr_test.go:85: STEP 9: Verify that istio-ca-root-cert created in Istio and member namespaces istio_csr_test.go:87: SUCCESS: istio-ca-root-cert created or updated in istio-system istio_csr_test.go:87: SUCCESS: istio-ca-root-cert created or updated in foo istio_csr_test.go:92: istio_csr_test.go:92: STEP 10: Verify that istio-ca-root-cert not created in non-member namespaces istio_csr_test.go:93: SUCCESS: istio-ca-root-cert not created in bar istio_csr_test.go:99: istio_csr_test.go:99: STEP 11: Deploy httpbin and sleep istio_csr_test.go:100: Install app "httpbin" in namespace "foo" istio_csr_test.go:100: Install app "sleep" in namespace "foo" istio_csr_test.go:100: Wait for app foo/httpbin to be ready istio_csr_test.go:100: Wait for app foo/sleep to be ready istio_csr_test.go:102: istio_csr_test.go:102: STEP 12: Check if httpbin returns 200 OK istio_csr_test.go:104: SUCCESS: Got expected 200 OK from httpbin istio_csr_test.go:114: istio_csr_test.go:114: STEP 13: Check mTLS traffic from ingress gateway to httpbin istio_csr_test.go:118: SUCCESS: received expected status code 200 test.go:77: test.go:82: Test completed in 139.03s (excluding cleanup) istio_csr_test.go:52: istio_csr_test.go:52: Performing cleanup istio_csr_test.go:58: Deleting namespaces: [foo] istio_csr_test.go:58: Creating namespaces: [foo] istio_csr_test.go:52: Cleanup completed in 31.45s --- PASS: TestIstioCsr (170.47s) === RUN TestPluginCaCert install.go:32: cert-manager-operator is not installed, starting installation install.go:59: install.go:59: STEP 1: Create namespace for cert-manager-operator install.go:60: Creating namespaces: [cert-manager-operator] install.go:62: install.go:62: STEP 2: Install cert-manager-operator install.go:67: Waiting for cert-manager-operator to succeed install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 1/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 2/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 3/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 4/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 5/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 6/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 7/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 8/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 9/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 10/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 11/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 12/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 13/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 14/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 15/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 16/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 17/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 18/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 19/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 20/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 21/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 22/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 23/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 24/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 25/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 26/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 27/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 28/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 29/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 30/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 31/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 32/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 33/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 34/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 35/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 36/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 37/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 38/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 39/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 40/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 41/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 42/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 43/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 44/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 45/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 46/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 47/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 48/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 49/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 50/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 51/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 52/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 53/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 54/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 55/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 56/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 57/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 58/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 59/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: Last attempt (60/60) failed. install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: FATAL: Command failed: oc describe -n cert-manager-operator csv/cert-manager-operator.v1.11.1 Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 test.go:77: test.go:79: Test failed in 74.11s (excluding cleanup) test.go:95: Capturing cluster state using must-gather registry.redhat.io/openshift-service-mesh/istio-must-gather-rhel8:2.4 test.go:99: /go/src/maistra-test-tool/tests/result-20230802171548/v2.4/failures-must-gather/20230802173544-TestPluginCaCert plugin_ca_test.go:48: plugin_ca_test.go:48: Performing cleanup plugin_ca_test.go:52: Deleting namespaces: [foo] plugin_ca_test.go:52: Creating namespaces: [foo] install.go:49: Deleting namespaces: [cert-manager-operator] install.go:50: Deleting namespaces: [cert-manager] plugin_ca_test.go:48: Cleanup completed in 31.03s --- FAIL: TestPluginCaCert (145.94s) FAIL FAIL github.com/maistra/maistra-test-tool/pkg/tests/tasks/security/certmanager 316.833s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestCircuitBreaking circuit_breaking_test.go:37: This test checks whether the circuit breaker functions correctly. Check documentation: https://istio.io/latest/docs/tasks/traffic-management/circuit-breaking/ circuit_breaking_test.go:44: circuit_breaking_test.go:44: STEP 1: Apply default SMCP and SMMR manifests circuit_breaking_test.go:44: Wait for condition condition=Ready on smcp istio-system/basic... circuit_breaking_test.go:44: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 circuit_breaking_test.go:44: Wait for condition condition=Ready on smcp istio-system/basic... circuit_breaking_test.go:44: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 circuit_breaking_test.go:44: Wait for condition condition=Ready on smcp istio-system/basic... circuit_breaking_test.go:44: SUCCESS: Condition condition=Ready met by smcp istio-system/basic circuit_breaking_test.go:46: circuit_breaking_test.go:46: STEP 2: Install httpbin and fortio circuit_breaking_test.go:47: Install app "httpbin" in namespace "bookinfo" circuit_breaking_test.go:47: Install app "fortio" in namespace "bookinfo" circuit_breaking_test.go:47: Wait for app bookinfo/httpbin to be ready circuit_breaking_test.go:47: Wait for app bookinfo/fortio to be ready circuit_breaking_test.go:49: circuit_breaking_test.go:49: STEP 3: Configure circuit breaker destination rule circuit_breaking_test.go:52: circuit_breaking_test.go:52: STEP 4: Verify connection with curl: expect 200 OK circuit_breaking_test.go:55: SUCCESS: Got expected 200 OK response from httpbin circuit_breaking_test.go:66: circuit_breaking_test.go:66: STEP 5: Trip the circuit breaker by sending 50 requests to httpbin with 2 connections circuit_breaking_test.go:67: We expect request with response code 503 circuit_breaking_test.go:79: Success rate 200: 46% circuit_breaking_test.go:80: Success rate 503: 54% circuit_breaking_test.go:85: circuit_breaking_test.go:85: STEP 6: Validate the circuit breaker is tripped by checking the istio-proxy log circuit_breaking_test.go:86: Verify istio-proxy pilot-agent stats, expected upstream_rq_pending_overflow value to be more than zero circuit_breaking_test.go:122: SUCCESS: Found Upstream_rq_pending_overflow : 27 test.go:77: test.go:82: Test completed in 32.12s (excluding cleanup) circuit_breaking_test.go:40: circuit_breaking_test.go:40: Performing cleanup circuit_breaking_test.go:41: Deleting namespaces: [bookinfo] circuit_breaking_test.go:41: Creating namespaces: [bookinfo] circuit_breaking_test.go:40: Cleanup completed in 17.61s --- PASS: TestCircuitBreaking (49.73s) === RUN TestFaultInjection fault_injection_test.go:47: fault_injection_test.go:47: STEP 1: Apply default SMCP and SMMR manifests fault_injection_test.go:47: Wait for condition condition=Ready on smcp istio-system/basic... fault_injection_test.go:47: SUCCESS: Condition condition=Ready met by smcp istio-system/basic fault_injection_test.go:49: fault_injection_test.go:49: STEP 2: Install Bookinfo fault_injection_test.go:50: Install app "bookinfo" in namespace "bookinfo" fault_injection_test.go:50: Create Bookinfo Gateway fault_injection_test.go:50: Create Bookinfo Destination Rules (all) fault_injection_test.go:50: Create Bookinfo Deployments fault_injection_test.go:50: Wait for app bookinfo/bookinfo to be ready fault_injection_test.go:52: Logging into bookinfo as "jason" === RUN TestFaultInjection/ratings-fault-delay fault_injection_test.go:60: fault_injection_test.go:60: STEP 1: check if productpage shows 'error fetching product reviews' due to delay injection fault_injection_test.go:62: FAILURE: expected request duration to be between 4s and 14s, but was 225ms fault_injection_test.go:62: SUCCESS: productpage shows 'error fetching product reviews', which is expected fault_injection_test.go:61: --- Attempt 1/60 failed. Retrying... fault_injection_test.go:62: FAILURE: expected request duration to be between 4s and 14s, but was 117ms fault_injection_test.go:62: SUCCESS: productpage shows 'error fetching product reviews', which is expected fault_injection_test.go:61: --- Attempt 2/60 failed. Retrying... fault_injection_test.go:62: FAILURE: expected request duration to be between 4s and 14s, but was 126ms fault_injection_test.go:62: SUCCESS: productpage shows 'error fetching product reviews', which is expected fault_injection_test.go:61: --- Attempt 3/60 failed. Retrying... fault_injection_test.go:62: SUCCESS: request completed in 6.033s, which is within the expected range 4s - 14s fault_injection_test.go:62: SUCCESS: productpage shows 'error fetching product reviews', which is expected fault_injection_test.go:61: --- Attempt 4/60 successful; total time: 10.04s subtest.go:23: subtest.go:28: Subtest completed in 10.45s (excluding cleanup) --- PASS: TestFaultInjection/ratings-fault-delay (10.45s) === RUN TestFaultInjection/ratings-fault-abort fault_injection_test.go:77: fault_injection_test.go:77: STEP 1: check if productpage shows ratings service as unavailable due to abort injection fault_injection_test.go:79: SUCCESS: productpage shows 'ratings service is currently unavailable' as expected subtest.go:23: subtest.go:28: Subtest completed in 0.62s (excluding cleanup) --- PASS: TestFaultInjection/ratings-fault-abort (0.62s) test.go:77: test.go:82: Test completed in 26.17s (excluding cleanup) fault_injection_test.go:43: fault_injection_test.go:43: Performing cleanup fault_injection_test.go:44: Deleting namespaces: [bookinfo] fault_injection_test.go:44: Creating namespaces: [bookinfo] fault_injection_test.go:43: Cleanup completed in 12.41s --- PASS: TestFaultInjection (38.58s) === RUN TestRequestRouting request_routing_test.go:37: request_routing_test.go:37: STEP 1: Apply default SMCP and SMMR manifests request_routing_test.go:37: Wait for condition condition=Ready on smcp istio-system/basic... request_routing_test.go:37: SUCCESS: Condition condition=Ready met by smcp istio-system/basic request_routing_test.go:39: request_routing_test.go:39: STEP 2: Install Bookinfo request_routing_test.go:40: Install app "bookinfo" in namespace "bookinfo" request_routing_test.go:40: Create Bookinfo Gateway request_routing_test.go:40: Create Bookinfo Destination Rules (all) request_routing_test.go:40: Create Bookinfo Deployments request_routing_test.go:40: Wait for app bookinfo/bookinfo to be ready request_routing_test.go:43: Logging into bookinfo as "jason" === RUN TestRequestRouting/not-logged-in request_routing_test.go:48: request_routing_test.go:48: STEP 1: get productpage without logging in; expect to get reviews-v1 (5x) request_routing_test.go:51: FAILURE: expected productpage to call reviews-v1, but got an unexpected response; expected the response to match file "productpage-normal-user-v1.html", but it matched "productpage-review-timeout.html" request_routing_test.go:49: --- Attempt 1/60 failed. Retrying... request_routing_test.go:51: FAILURE: expected productpage to call reviews-v1, but got an unexpected response; expected the response to match file "productpage-normal-user-v1.html", but it matched "productpage-review-timeout.html" request_routing_test.go:49: --- Attempt 2/60 failed. Retrying... request_routing_test.go:51: FAILURE: expected productpage to call reviews-v1, but got an unexpected response; expected the response to match file "productpage-normal-user-v1.html", but it matched "productpage-review-timeout.html" request_routing_test.go:49: --- Attempt 3/60 failed. Retrying... request_routing_test.go:51: FAILURE: expected productpage to call reviews-v1, but got an unexpected response; expected the response to match file "productpage-normal-user-v1.html", but it matched "productpage-review-timeout.html" request_routing_test.go:49: --- Attempt 4/60 failed. Retrying... request_routing_test.go:51: FAILURE: expected productpage to call reviews-v1, but got an unexpected response; expected the response to match file "productpage-normal-user-v1.html", but it matched "productpage-review-timeout.html" request_routing_test.go:49: --- Attempt 5/60 failed. Retrying... request_routing_test.go:51: FAILURE: expected productpage to call reviews-v1, but got an unexpected response; expected the response to match file "productpage-normal-user-v1.html", but it matched "productpage-review-timeout.html" request_routing_test.go:49: --- Attempt 6/60 failed. Retrying... request_routing_test.go:51: SUCCESS: productpage called reviews-v1 request_routing_test.go:51: SUCCESS: productpage called reviews-v1 request_routing_test.go:51: SUCCESS: productpage called reviews-v1 request_routing_test.go:51: SUCCESS: productpage called reviews-v1 request_routing_test.go:51: SUCCESS: productpage called reviews-v1 request_routing_test.go:49: --- Attempt 7/60 successful; total time: 7.76s subtest.go:23: subtest.go:28: Subtest completed in 8.35s (excluding cleanup) --- PASS: TestRequestRouting/not-logged-in (8.35s) === RUN TestRequestRouting/logged-in request_routing_test.go:65: request_routing_test.go:65: STEP 1: get productpage as logged-in user; expect to get reviews-v2 (5x) request_routing_test.go:68: FAILURE: expected productpage to call reviews-v2, but got an unexpected response; expected the response to match file "productpage-test-user-v2.html", but it matched "productpage-test-user-v1.html" request_routing_test.go:66: --- Attempt 1/60 failed. Retrying... request_routing_test.go:68: SUCCESS: productpage called reviews-v2 request_routing_test.go:68: SUCCESS: productpage called reviews-v2 request_routing_test.go:68: SUCCESS: productpage called reviews-v2 request_routing_test.go:68: SUCCESS: productpage called reviews-v2 request_routing_test.go:68: SUCCESS: productpage called reviews-v2 request_routing_test.go:66: --- Attempt 2/60 successful; total time: 1.75s subtest.go:23: subtest.go:28: Subtest completed in 2.12s (excluding cleanup) --- PASS: TestRequestRouting/logged-in (2.12s) test.go:77: test.go:82: Test completed in 24.66s (excluding cleanup) request_routing_test.go:33: request_routing_test.go:33: Performing cleanup request_routing_test.go:34: Deleting namespaces: [bookinfo] request_routing_test.go:34: Creating namespaces: [bookinfo] request_routing_test.go:33: Cleanup completed in 12.49s --- PASS: TestRequestRouting (37.15s) === RUN TestRequestTimeouts request_timeouts_test.go:42: request_timeouts_test.go:42: STEP 1: Apply default SMCP and SMMR manifests request_timeouts_test.go:42: Wait for condition condition=Ready on smcp istio-system/basic... request_timeouts_test.go:42: SUCCESS: Condition condition=Ready met by smcp istio-system/basic request_timeouts_test.go:44: request_timeouts_test.go:44: STEP 2: Install Bookinfo request_timeouts_test.go:45: Install app "bookinfo" in namespace "bookinfo" request_timeouts_test.go:45: Create Bookinfo Gateway request_timeouts_test.go:45: Create Bookinfo Destination Rules (all) request_timeouts_test.go:45: Create Bookinfo Deployments request_timeouts_test.go:45: Wait for app bookinfo/bookinfo to be ready request_timeouts_test.go:51: request_timeouts_test.go:51: STEP 3: make sure there is no timeout before applying delay and timeout in VirtualServices request_timeouts_test.go:53: FAILURE: unexpected response; expected the response to match file "productpage-normal-user-v1.html", but it matched "productpage-review-timeout.html" request_timeouts_test.go:52: --- Attempt 1/60 failed. Retrying... request_timeouts_test.go:53: FAILURE: unexpected response; expected the response to match file "productpage-normal-user-v1.html", but it matched "productpage-review-timeout.html" request_timeouts_test.go:52: --- Attempt 2/60 failed. Retrying... request_timeouts_test.go:53: FAILURE: unexpected response; expected the response to match file "productpage-normal-user-v1.html", but it matched "productpage-review-timeout.html" request_timeouts_test.go:52: --- Attempt 3/60 failed. Retrying... request_timeouts_test.go:53: FAILURE: unexpected response; expected the response to match file "productpage-normal-user-v1.html", but it matched "productpage-review-timeout.html" request_timeouts_test.go:52: --- Attempt 4/60 failed. Retrying... request_timeouts_test.go:53: FAILURE: unexpected response; expected the response to match file "productpage-normal-user-v1.html", but it matched "productpage-review-timeout.html" request_timeouts_test.go:52: --- Attempt 5/60 failed. Retrying... request_timeouts_test.go:53: SUCCESS: received normal productpage response request_timeouts_test.go:52: --- Attempt 6/60 successful; total time: 6.36s request_timeouts_test.go:62: request_timeouts_test.go:62: STEP 4: apply delay and timeout in VirtualServices request_timeouts_test.go:65: request_timeouts_test.go:65: STEP 5: check if productpage shows 'error fetching product reviews' due to delay and timeout injection request_timeouts_test.go:68: FAILURE: expected productpage to show 'error fetching product reviews', but got a different response; expected the response to match file "productpage-review-timeout.html", but it matched "productpage-normal-user-v1.html" request_timeouts_test.go:66: --- Attempt 1/60 failed. Retrying... request_timeouts_test.go:68: SUCCESS: productpage shows 'error fetching product reviews', which is expected request_timeouts_test.go:68: SUCCESS: productpage shows 'error fetching product reviews', which is expected request_timeouts_test.go:68: SUCCESS: productpage shows 'error fetching product reviews', which is expected request_timeouts_test.go:68: SUCCESS: productpage shows 'error fetching product reviews', which is expected request_timeouts_test.go:68: SUCCESS: productpage shows 'error fetching product reviews', which is expected request_timeouts_test.go:68: SUCCESS: productpage shows 'error fetching product reviews', which is expected request_timeouts_test.go:66: --- Attempt 2/60 successful; total time: 7.19s test.go:77: test.go:82: Test completed in 28.25s (excluding cleanup) request_timeouts_test.go:38: request_timeouts_test.go:38: Performing cleanup request_timeouts_test.go:39: Deleting namespaces: [bookinfo] request_timeouts_test.go:39: Creating namespaces: [bookinfo] request_timeouts_test.go:38: Cleanup completed in 12.94s --- PASS: TestRequestTimeouts (41.19s) === RUN TestMirroring traffic_mirroring_test.go:38: traffic_mirroring_test.go:38: STEP 1: Apply default SMCP and SMMR manifests traffic_mirroring_test.go:38: Wait for condition condition=Ready on smcp istio-system/basic... traffic_mirroring_test.go:38: SUCCESS: Condition condition=Ready met by smcp istio-system/basic traffic_mirroring_test.go:40: traffic_mirroring_test.go:40: STEP 2: Install httpbin-v1, httpbin-v2, and sleep traffic_mirroring_test.go:41: Install app "httpbin-v1" in namespace "bookinfo" traffic_mirroring_test.go:41: Install app "httpbin-v2" in namespace "bookinfo" traffic_mirroring_test.go:41: Install app "sleep" in namespace "bookinfo" traffic_mirroring_test.go:41: Wait for app bookinfo/httpbin-v1 to be ready traffic_mirroring_test.go:41: Wait for app bookinfo/httpbin-v2 to be ready traffic_mirroring_test.go:41: Wait for app bookinfo/sleep to be ready === RUN TestMirroring/no_mirroring traffic_mirroring_test.go:49: traffic_mirroring_test.go:49: STEP 1: sending HTTP request from sleep to httpbin-v1, not expecting mirroring to v2 traffic_mirroring_test.go:58: SUCCESS: request received by httpbin-v1 traffic_mirroring_test.go:66: SUCCESS: request not mirrored to httpbin-v2 subtest.go:23: subtest.go:28: Subtest completed in 1.42s (excluding cleanup) --- PASS: TestMirroring/no_mirroring (1.42s) === RUN TestMirroring/mirroring_to_httpbin-v2 traffic_mirroring_test.go:79: traffic_mirroring_test.go:79: STEP 1: sending HTTP request from sleep to httpbin-v1, expecting mirroring to v2 traffic_mirroring_test.go:88: SUCCESS: request received by httpbin-v1 traffic_mirroring_test.go:96: SUCCESS: request mirrored to httpbin-v2 subtest.go:23: subtest.go:28: Subtest completed in 1.42s (excluding cleanup) --- PASS: TestMirroring/mirroring_to_httpbin-v2 (1.42s) test.go:77: test.go:82: Test completed in 15.67s (excluding cleanup) traffic_mirroring_test.go:34: traffic_mirroring_test.go:34: Performing cleanup traffic_mirroring_test.go:35: Deleting namespaces: [bookinfo] traffic_mirroring_test.go:35: Creating namespaces: [bookinfo] traffic_mirroring_test.go:34: Cleanup completed in 17.75s --- PASS: TestMirroring (33.42s) === RUN TestTrafficShifting traffic_shifting_test.go:41: traffic_shifting_test.go:41: STEP 1: Apply default SMCP and SMMR manifests traffic_shifting_test.go:41: Wait for condition condition=Ready on smcp istio-system/basic... traffic_shifting_test.go:41: SUCCESS: Condition condition=Ready met by smcp istio-system/basic traffic_shifting_test.go:43: traffic_shifting_test.go:43: STEP 2: Install Bookinfo traffic_shifting_test.go:44: Install app "bookinfo" in namespace "bookinfo" traffic_shifting_test.go:44: Create Bookinfo Gateway traffic_shifting_test.go:44: Create Bookinfo Destination Rules (all) traffic_shifting_test.go:44: Create Bookinfo Deployments traffic_shifting_test.go:44: Wait for app bookinfo/bookinfo to be ready === RUN TestTrafficShifting/50_percent_to_v3 traffic_shifting_test.go:50: traffic_shifting_test.go:50: STEP 1: configure VirtualService to split traffic 50% to v1 and 50% to v3 traffic_shifting_test.go:53: traffic_shifting_test.go:53: STEP 2: Make 100 requests and check if v1 and v3 get 50% of requests each (tolerance: 20%) traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:105: FAILURE: Response did not match any expected value, but matched file "productpage-review-timeout.html" traffic_shifting_test.go:54: --- Attempt 1/60 failed. Retrying... traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:105: FAILURE: Response did not match any expected value, but matched file "productpage-review-timeout.html" traffic_shifting_test.go:54: --- Attempt 2/60 failed. Retrying... traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:105: FAILURE: Response did not match any expected value, but matched file "productpage-review-timeout.html" traffic_shifting_test.go:54: --- Attempt 3/60 failed. Retrying... traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:105: FAILURE: Response did not match any expected value, but matched file "productpage-review-timeout.html" traffic_shifting_test.go:54: --- Attempt 4/60 failed. Retrying... traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:116: SUCCESS: 54/100 responses matched productpage-normal-user-v3.html (actual rate 0.540000, expected 0.500000, tolerance 0.200000) traffic_shifting_test.go:116: SUCCESS: 46/100 responses matched productpage-normal-user-v1.html (actual rate 0.460000, expected 0.500000, tolerance 0.200000) traffic_shifting_test.go:54: --- Attempt 5/60 successful; total time: 12.63s subtest.go:23: subtest.go:28: Subtest completed in 12.99s (excluding cleanup) --- PASS: TestTrafficShifting/50_percent_to_v3 (12.99s) === RUN TestTrafficShifting/100_percent_to_v3 traffic_shifting_test.go:64: traffic_shifting_test.go:64: STEP 1: configure VirtualService to send all traffic to v3 traffic_shifting_test.go:67: traffic_shifting_test.go:67: STEP 2: Make 100 requests and check if all of them go to v3 (tolerance: 0%) traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:118: FAILURE: 1/100 responses matched productpage-normal-user-v1.html (actual rate 0.010000, expected 0.000000, tolerance 0.000000) traffic_shifting_test.go:118: FAILURE: 99/100 responses matched productpage-normal-user-v3.html (actual rate 0.990000, expected 1.000000, tolerance 0.000000) traffic_shifting_test.go:68: --- Attempt 1/60 failed. Retrying... traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:82: SUCCESS: received expected status code 200 traffic_shifting_test.go:116: SUCCESS: 100/100 responses matched productpage-normal-user-v3.html (actual rate 1.000000, expected 1.000000, tolerance 0.000000) traffic_shifting_test.go:68: --- Attempt 2/60 successful; total time: 15.19s subtest.go:23: subtest.go:28: Subtest completed in 15.61s (excluding cleanup) --- PASS: TestTrafficShifting/100_percent_to_v3 (15.61s) test.go:77: test.go:82: Test completed in 43.17s (excluding cleanup) traffic_shifting_test.go:37: traffic_shifting_test.go:37: Performing cleanup traffic_shifting_test.go:38: Deleting namespaces: [bookinfo] traffic_shifting_test.go:38: Creating namespaces: [bookinfo] traffic_shifting_test.go:37: Cleanup completed in 12.65s --- PASS: TestTrafficShifting (55.82s) === RUN TestTcpTrafficShifting traffic_tcp_shifting_test.go:41: This test validates traffic shifting for TCP traffic. traffic_tcp_shifting_test.go:42: Doc reference: https://istio.io/v1.14/docs/tasks/traffic-management/tcp-traffic-shifting/ traffic_tcp_shifting_test.go:44: traffic_tcp_shifting_test.go:44: STEP 1: Apply default SMCP and SMMR manifests traffic_tcp_shifting_test.go:44: Wait for condition condition=Ready on smcp istio-system/basic... traffic_tcp_shifting_test.go:44: SUCCESS: Condition condition=Ready met by smcp istio-system/basic traffic_tcp_shifting_test.go:46: traffic_tcp_shifting_test.go:46: STEP 2: Install sleep, echoV1 and echoV2 traffic_tcp_shifting_test.go:47: Install app "sleep" in namespace "foo" traffic_tcp_shifting_test.go:47: Install app "echo-v1" in namespace "foo" traffic_tcp_shifting_test.go:47: Install app "echo-v2" in namespace "foo" traffic_tcp_shifting_test.go:47: Wait for app foo/sleep to be ready traffic_tcp_shifting_test.go:47: Wait for app foo/echo-v1 to be ready traffic_tcp_shifting_test.go:47: Wait for app foo/echo-v2 to be ready === RUN TestTcpTrafficShifting/tcp_shift_100_percent_to_v1 traffic_tcp_shifting_test.go:54: traffic_tcp_shifting_test.go:54: STEP 1: Shifting all TCP traffic to v1 traffic_tcp_shifting_test.go:57: traffic_tcp_shifting_test.go:57: STEP 2: make 20 requests and checking if all of them go to v1 (tolerance: 0%) traffic_tcp_shifting_test.go:114: SUCCESS: 20/20 responses matched one (actual rate 1.000000, expected 1.000000, tolerance 0.000000) subtest.go:23: subtest.go:28: Subtest completed in 0.94s (excluding cleanup) traffic_tcp_shifting_test.go:50: traffic_tcp_shifting_test.go:50: Performing cleanup traffic_tcp_shifting_test.go:50: Cleanup completed in 0.20s --- PASS: TestTcpTrafficShifting/tcp_shift_100_percent_to_v1 (1.14s) === RUN TestTcpTrafficShifting/tcp_shift_20_percent_to_v2 traffic_tcp_shifting_test.go:72: traffic_tcp_shifting_test.go:72: STEP 1: Shifting 20 percent TCP traffic to v2 traffic_tcp_shifting_test.go:75: traffic_tcp_shifting_test.go:75: STEP 2: make 100 requests and checking if 20 percent of them go to v2 (tolerance: 10%) traffic_tcp_shifting_test.go:114: SUCCESS: 80/100 responses matched one (actual rate 0.800000, expected 0.800000, tolerance 0.100000) traffic_tcp_shifting_test.go:114: SUCCESS: 20/100 responses matched two (actual rate 0.200000, expected 0.200000, tolerance 0.100000) subtest.go:23: subtest.go:28: Subtest completed in 1.15s (excluding cleanup) traffic_tcp_shifting_test.go:68: traffic_tcp_shifting_test.go:68: Performing cleanup traffic_tcp_shifting_test.go:68: Cleanup completed in 0.19s --- PASS: TestTcpTrafficShifting/tcp_shift_20_percent_to_v2 (1.34s) test.go:77: test.go:82: Test completed in 14.98s (excluding cleanup) traffic_tcp_shifting_test.go:37: traffic_tcp_shifting_test.go:37: Performing cleanup traffic_tcp_shifting_test.go:37: Cleanup completed in 0.64s --- PASS: TestTcpTrafficShifting (15.62s) PASS ok github.com/maistra/maistra-test-tool/pkg/tests/tasks/traffic 271.964s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestAccessExternalServices access_external_services_test.go:43: This test validates accesses to external services access_external_services_test.go:45: access_external_services_test.go:45: STEP 1: Apply default SMCP and SMMR manifests access_external_services_test.go:45: Wait for condition condition=Ready on smcp istio-system/basic... access_external_services_test.go:45: SUCCESS: Condition condition=Ready met by smcp istio-system/basic access_external_services_test.go:47: access_external_services_test.go:47: STEP 2: Install sleep into bookinfo access_external_services_test.go:49: Install app "sleep" in namespace "bookinfo" access_external_services_test.go:49: Wait for app bookinfo/sleep to be ready access_external_services_test.go:51: access_external_services_test.go:51: STEP 3: Install httpbin in mesh-external access_external_services_test.go:53: Install app "httpbin" in namespace "mesh-external" access_external_services_test.go:53: Wait for app mesh-external/httpbin to be ready access_external_services_test.go:55: access_external_services_test.go:55: STEP 4: Make request to external httpbin from sleep egress_common.go:24: SUCCESS: Got expected 200 OK from http://httpbin.mesh-external:8000/headers access_external_services_test.go:59: access_external_services_test.go:59: STEP 5: Make sure that external httpbin was not discovered by Istio istioctl.go:12: SUCCESS: Httpbin was not discovered access_external_services_test.go:67: access_external_services_test.go:67: STEP 6: Patch outbound traffic policy to registry only - see https://istio.io/latest/docs/tasks/traffic-management/egress/egress-control/#envoy-passthrough-to-external-services access_external_services_test.go:81: access_external_services_test.go:81: STEP 7: Make request to external httpbin from sleep again, and expect it denied egress_common.go:31: FAILURE: Expect request to failed, but got a response; expected to find the string 'CURL_FAILED' in the output, but it wasn't found egress_common.go:31: --- Attempt 1/60 failed. Retrying... egress_common.go:31: SUCCESS: Got a failure message as expected egress_common.go:31: --- Attempt 2/60 successful; total time: 1.87s === RUN TestAccessExternalServices/allow_request_to_external_httpbin_after_applying_ServiceEntry access_external_services_test.go:89: access_external_services_test.go:89: STEP 1: Apply a ServiceEntry for external httpbin access_external_services_test.go:92: access_external_services_test.go:92: STEP 2: Send a request to external httpbin egress_common.go:24: SUCCESS: Got expected 200 OK from http://httpbin.mesh-external:8000/headers subtest.go:23: subtest.go:28: Subtest completed in 0.82s (excluding cleanup) access_external_services_test.go:85: access_external_services_test.go:85: Performing cleanup access_external_services_test.go:85: Cleanup completed in 0.13s --- PASS: TestAccessExternalServices/allow_request_to_external_httpbin_after_applying_ServiceEntry (0.96s) test.go:77: test.go:82: Test completed in 17.34s (excluding cleanup) access_external_services_test.go:35: access_external_services_test.go:35: Performing cleanup access_external_services_test.go:35: Cleanup completed in 0.56s --- PASS: TestAccessExternalServices (17.90s) === RUN TestEgressGateways egress_gateways_test.go:33: egress_gateways_test.go:33: STEP 1: Apply default SMCP and SMMR manifests egress_gateways_test.go:33: Wait for condition condition=Ready on smcp istio-system/basic... egress_gateways_test.go:33: SUCCESS: Condition condition=Ready met by smcp istio-system/basic egress_gateways_test.go:35: egress_gateways_test.go:35: STEP 2: Install sleep pod egress_gateways_test.go:37: Install app "sleep" in namespace "bookinfo" egress_gateways_test.go:37: Wait for app bookinfo/sleep to be ready === RUN TestEgressGateways/HTTP egress_gateways_test.go:40: egress_gateways_test.go:40: STEP 1: Install external httpbin egress_gateways_test.go:42: Install app "httpbin" in namespace "mesh-external" egress_gateways_test.go:42: Wait for app mesh-external/httpbin to be ready egress_gateways_test.go:44: egress_gateways_test.go:44: STEP 2: Apply a ServiceEntry for external httpbin egress_gateways_test.go:50: egress_gateways_test.go:50: STEP 3: Apply a gateway and virtual service for external httpbin egress_common.go:24: SUCCESS: Got expected 200 OK from http://httpbin.mesh-external:8000/headers subtest.go:23: subtest.go:28: Subtest completed in 1.83s (excluding cleanup) egress_gateways_test.go:52: egress_gateways_test.go:52: Performing cleanup egress_gateways_test.go:52: Cleanup completed in 0.20s egress_gateways_test.go:46: egress_gateways_test.go:46: Performing cleanup egress_gateways_test.go:46: Cleanup completed in 0.12s --- PASS: TestEgressGateways/HTTP (2.15s) === RUN TestEgressGateways/HTTPS egress_gateways_test.go:60: egress_gateways_test.go:60: STEP 1: Install external nginx egress_gateways_test.go:61: Install app "nginx" in namespace "mesh-external" egress_gateways_test.go:61: Wait for app mesh-external/nginx to be ready egress_gateways_test.go:63: egress_gateways_test.go:63: STEP 2: Create ServiceEntry for external nginx, port 80 and 443 egress_gateways_test.go:69: egress_gateways_test.go:69: STEP 3: Create a TLS ServiceEntry to external nginx egress_gateways_test.go:75: egress_gateways_test.go:75: STEP 4: Create a https Gateway to external nginx egress_gateways_test.go:81: Send HTTPS request to external nginx egress_common.go:39: SUCCESS: Got expected 200 OK from curl -sSL --insecure -o /dev/null -w "%{http_code}" https://my-nginx.mesh-external.svc.cluster.local 2>/dev/null || echo CURL_FAILED subtest.go:23: subtest.go:28: Subtest completed in 6.94s (excluding cleanup) egress_gateways_test.go:77: egress_gateways_test.go:77: Performing cleanup egress_gateways_test.go:77: Cleanup completed in 0.18s egress_gateways_test.go:71: egress_gateways_test.go:71: Performing cleanup egress_gateways_test.go:71: Cleanup completed in 0.12s egress_gateways_test.go:65: egress_gateways_test.go:65: Performing cleanup egress_gateways_test.go:65: Cleanup completed in 0.12s --- PASS: TestEgressGateways/HTTPS (7.35s) test.go:77: test.go:82: Test completed in 21.79s (excluding cleanup) egress_gateways_test.go:29: egress_gateways_test.go:29: Performing cleanup egress_gateways_test.go:30: Deleting namespaces: [bookinfo] egress_gateways_test.go:30: Creating namespaces: [bookinfo] egress_gateways_test.go:29: Cleanup completed in 12.06s --- PASS: TestEgressGateways (33.85s) === RUN TestTLSOrigination egress_gateways_tls_file_mount_test.go:33: This test verifies that TLS origination works in 2 scenarios: egress_gateways_tls_file_mount_test.go:34: 1) Egress gateway TLS Origination egress_gateways_tls_file_mount_test.go:35: 2) MTLS Origination with file mount (certificates mounted in egress gateway pod) egress_gateways_tls_file_mount_test.go:41: egress_gateways_tls_file_mount_test.go:41: STEP 1: Apply default SMCP and SMMR manifests egress_gateways_tls_file_mount_test.go:41: Wait for condition condition=Ready on smcp istio-system/basic... egress_gateways_tls_file_mount_test.go:41: SUCCESS: Condition condition=Ready met by smcp istio-system/basic egress_gateways_tls_file_mount_test.go:43: egress_gateways_tls_file_mount_test.go:43: STEP 2: Install sleep pod egress_gateways_tls_file_mount_test.go:44: Install app "sleep" in namespace "bookinfo" egress_gateways_tls_file_mount_test.go:44: Wait for app bookinfo/sleep to be ready === RUN TestTLSOrigination/Egress_Gateway_without_file_mount egress_gateways_tls_file_mount_test.go:47: Perform TLS origination with an egress gateway egress_gateways_tls_file_mount_test.go:49: egress_gateways_tls_file_mount_test.go:49: STEP 1: Install external nginx egress_gateways_tls_file_mount_test.go:50: Install app "nginx" in namespace "mesh-external" egress_gateways_tls_file_mount_test.go:50: Wait for app mesh-external/nginx to be ready egress_gateways_tls_file_mount_test.go:52: egress_gateways_tls_file_mount_test.go:52: STEP 2: Make sure that mesh external namespace is not discovered by Istio - it would happen if mesh-external namespaces was added to the SMMR istioctl.go:12: SUCCESS: mesh-external namespace was not discovered egress_gateways_tls_file_mount_test.go:60: egress_gateways_tls_file_mount_test.go:60: STEP 3: Create ServiceEntry for external nginx, port 80 and 443 egress_gateways_tls_file_mount_test.go:66: egress_gateways_tls_file_mount_test.go:66: STEP 4: Create a Gateway, DestinationRule, and VirtualService to route requests to external nginx through the egress gateway egress_gateways_tls_file_mount_test.go:74: egress_gateways_tls_file_mount_test.go:74: STEP 5: Verify that request to external nginx is routed through the egress gateway (response 200 indicates that the TLS origination is done by the egress gateway) egress_gateways_tls_file_mount_test.go:75: SUCCESS: Get expected response: Welcome to nginx subtest.go:23: subtest.go:28: Subtest completed in 3.52s (excluding cleanup) egress_gateways_tls_file_mount_test.go:69: egress_gateways_tls_file_mount_test.go:69: Performing cleanup egress_gateways_tls_file_mount_test.go:69: Cleanup completed in 0.32s egress_gateways_tls_file_mount_test.go:62: egress_gateways_tls_file_mount_test.go:62: Performing cleanup egress_gateways_tls_file_mount_test.go:62: Cleanup completed in 0.15s --- PASS: TestTLSOrigination/Egress_Gateway_without_file_mount (3.98s) === RUN TestTLSOrigination/mTLS_with_file_mount egress_gateways_tls_file_mount_test.go:84: Perform mTLS origination with an egress gateway egress_gateways_tls_file_mount_test.go:98: egress_gateways_tls_file_mount_test.go:98: STEP 1: Deploy nginx mTLS server and create secrets in the mesh namespace egress_gateways_tls_file_mount_test.go:103: Install app "nginx" in namespace "mesh-external" egress_gateways_tls_file_mount_test.go:105: egress_gateways_tls_file_mount_test.go:105: STEP 2: Patch egress gateway with File Mount configuration egress_gateways_tls_file_mount_test.go:108: egress_gateways_tls_file_mount_test.go:108: STEP 3: Configure mTLS origination for egress traffic egress_gateways_tls_file_mount_test.go:112: egress_gateways_tls_file_mount_test.go:112: STEP 4: Wait for egress gateway and nginx to be ready egress_gateways_tls_file_mount_test.go:114: Wait for app mesh-external/nginx to be ready egress_gateways_tls_file_mount_test.go:116: egress_gateways_tls_file_mount_test.go:116: STEP 5: Verify NGINX server egress_gateways_tls_file_mount_test.go:117: SUCCESS: Get expected response: Welcome to nginx subtest.go:23: subtest.go:28: Subtest completed in 3.75s (excluding cleanup) egress_gateways_tls_file_mount_test.go:85: egress_gateways_tls_file_mount_test.go:85: Performing cleanup egress_gateways_tls_file_mount_test.go:95: SUCCESS: Pod istio-ingressgateway-59d64db9d9-knb6d is being deleted egress_gateways_tls_file_mount_test.go:85: Cleanup completed in 7.32s --- PASS: TestTLSOrigination/mTLS_with_file_mount (11.07s) test.go:77: test.go:82: Test completed in 26.57s (excluding cleanup) egress_gateways_tls_file_mount_test.go:37: egress_gateways_tls_file_mount_test.go:37: Performing cleanup egress_gateways_tls_file_mount_test.go:37: Cleanup completed in 0.40s --- PASS: TestTLSOrigination (26.98s) === RUN TestTLSOriginationSDS egress_gateways_tls_sds_test.go:37: Perform mTLS origination with an egress gateway egress_gateways_tls_sds_test.go:38: egress_gateways_tls_sds_test.go:38: STEP 1: Apply default SMCP and SMMR manifests egress_gateways_tls_sds_test.go:38: Wait for condition condition=Ready on smcp istio-system/basic... egress_gateways_tls_sds_test.go:38: SUCCESS: Condition condition=Ready met by smcp istio-system/basic egress_gateways_tls_sds_test.go:40: egress_gateways_tls_sds_test.go:40: STEP 2: Install sleep pod egress_gateways_tls_sds_test.go:42: Install app "sleep" in namespace "bookinfo" egress_gateways_tls_sds_test.go:42: Wait for app bookinfo/sleep to be ready egress_gateways_tls_sds_test.go:44: egress_gateways_tls_sds_test.go:44: STEP 3: Deploy nginx mTLS server and create secrets in the mesh namespace egress_gateways_tls_sds_test.go:45: Install app "nginx" in namespace "mesh-external" egress_gateways_tls_sds_test.go:45: FATAL: Command failed: kubectl -n mesh-external delete secret nginx-ca-certs --ignore-not-found The connection to the server api.ci-op-11b996y7-8c5aa.cspilp.interop.ccitredhat.com:6443 was refused - did you specify the right host or port? error: exit status 1 test.go:77: test.go:79: Test failed in 12.72s (excluding cleanup) test.go:95: Capturing cluster state using must-gather registry.redhat.io/openshift-service-mesh/istio-must-gather-rhel8:2.4 test.go:99: /go/src/maistra-test-tool/tests/result-20230802171548/v2.4/failures-must-gather/20230802174300-TestTLSOriginationSDS egress_gateways_tls_sds_test.go:29: egress_gateways_tls_sds_test.go:29: Performing cleanup egress_gateways_tls_sds_test.go:30: Deleting namespaces: [bookinfo] egress_gateways_tls_sds_test.go:30: Creating namespaces: [bookinfo] egress_gateways_tls_sds_test.go:31: Deleting namespaces: [mesh-external] egress_gateways_tls_sds_test.go:31: Creating namespaces: [mesh-external] egress_gateways_tls_sds_test.go:29: Cleanup completed in 41.21s --- FAIL: TestTLSOriginationSDS (116.76s) === RUN TestEgressTLSOrigination egress_tls_origination_test.go:35: egress_tls_origination_test.go:35: STEP 1: Apply default SMCP and SMMR manifests egress_tls_origination_test.go:35: Wait for condition condition=Ready on smcp istio-system/basic... egress_tls_origination_test.go:35: SUCCESS: Condition condition=Ready met by smcp istio-system/basic egress_tls_origination_test.go:37: egress_tls_origination_test.go:37: STEP 2: Install sleep pod egress_tls_origination_test.go:38: Install app "sleep" in namespace "bookinfo" egress_tls_origination_test.go:38: Wait for app bookinfo/sleep to be ready === RUN TestEgressTLSOrigination/TrafficManagement_egress_tls_origination egress_tls_origination_test.go:41: TLS origination for egress traffic egress_tls_origination_test.go:49: Install app "nginx" in namespace "mesh-external" egress_tls_origination_test.go:49: Wait for app mesh-external/nginx to be ready egress_common.go:24: SUCCESS: Got expected 200 OK from http://my-nginx.mesh-external.svc.cluster.local subtest.go:23: subtest.go:28: Subtest completed in 6.91s (excluding cleanup) egress_tls_origination_test.go:42: egress_tls_origination_test.go:42: Performing cleanup egress_tls_origination_test.go:42: Cleanup completed in 0.99s --- PASS: TestEgressTLSOrigination/TrafficManagement_egress_tls_origination (7.89s) test.go:77: test.go:82: Test completed in 19.07s (excluding cleanup) egress_tls_origination_test.go:30: egress_tls_origination_test.go:30: Performing cleanup egress_tls_origination_test.go:31: Deleting namespaces: [mesh-external] egress_tls_origination_test.go:31: Creating namespaces: [mesh-external] egress_tls_origination_test.go:30: Cleanup completed in 6.65s --- PASS: TestEgressTLSOrigination (25.72s) === RUN TestEgressWildcard egress_wildcard_hosts_test.go:31: This test checks if the wildcard in the ServiceEntry and Gateway works as expected for Egress traffic. egress_wildcard_hosts_test.go:35: egress_wildcard_hosts_test.go:35: STEP 1: Apply default SMCP and SMMR manifests egress_wildcard_hosts_test.go:35: Wait for condition condition=Ready on smcp istio-system/basic... egress_wildcard_hosts_test.go:35: SUCCESS: Condition condition=Ready met by smcp istio-system/basic egress_wildcard_hosts_test.go:37: egress_wildcard_hosts_test.go:37: STEP 2: Install the sleep pod egress_wildcard_hosts_test.go:38: Install app "sleep" in namespace "bookinfo" egress_wildcard_hosts_test.go:38: Wait for app bookinfo/sleep to be ready === RUN TestEgressWildcard/ServiceEntry egress_wildcard_hosts_test.go:44: egress_wildcard_hosts_test.go:44: STEP 1: Configure ServiceEntry with wildcard host *.wikipedia.org egress_wildcard_hosts_test.go:66: egress_wildcard_hosts_test.go:66: STEP 2: Check external request to en.wikipedia.org and de.wikipedia.org egress_wildcard_hosts_test.go:68: SUCCESS: Received the correct response from en.wikipedia.org egress_wildcard_hosts_test.go:77: SUCCESS: Received the correct response from de.wikipedia.org subtest.go:23: subtest.go:28: Subtest completed in 1.55s (excluding cleanup) egress_wildcard_hosts_test.go:46: egress_wildcard_hosts_test.go:46: Performing cleanup egress_wildcard_hosts_test.go:46: Cleanup completed in 0.13s --- PASS: TestEgressWildcard/ServiceEntry (1.68s) === RUN TestEgressWildcard/Gateway egress_wildcard_hosts_test.go:54: egress_wildcard_hosts_test.go:54: STEP 1: Configure egress Gateway with wildcard host *.wikipedia.org egress_wildcard_hosts_test.go:66: egress_wildcard_hosts_test.go:66: STEP 2: Check external request to en.wikipedia.org and de.wikipedia.org egress_wildcard_hosts_test.go:68: SUCCESS: Received the correct response from en.wikipedia.org egress_wildcard_hosts_test.go:77: SUCCESS: Received the correct response from de.wikipedia.org subtest.go:23: subtest.go:28: Subtest completed in 1.69s (excluding cleanup) egress_wildcard_hosts_test.go:56: egress_wildcard_hosts_test.go:56: Performing cleanup egress_wildcard_hosts_test.go:56: Cleanup completed in 0.23s --- PASS: TestEgressWildcard/Gateway (1.92s) test.go:77: test.go:82: Test completed in 15.12s (excluding cleanup) egress_wildcard_hosts_test.go:39: egress_wildcard_hosts_test.go:39: Performing cleanup egress_wildcard_hosts_test.go:39: Cleanup completed in 0.31s --- PASS: TestEgressWildcard (15.43s) FAIL FAIL github.com/maistra/maistra-test-tool/pkg/tests/tasks/traffic/egress 237.086s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestGatewayApi gatewayapi_test.go:28: gatewayapi_test.go:28: STEP 1: Apply default SMCP and SMMR manifests gatewayapi_test.go:28: Wait for condition condition=Ready on smcp istio-system/basic... gatewayapi_test.go:28: SUCCESS: Condition condition=Ready met by smcp istio-system/basic gatewayapi_test.go:30: gatewayapi_test.go:30: STEP 2: Install Gateway API CRD's gatewayapi_test.go:33: Creating namespaces: [foo] === RUN TestGatewayApi/Deploy_the_Kubernetes_Gateway_API gatewayapi_test.go:42: gatewayapi_test.go:42: STEP 1: Install httpbin gatewayapi_test.go:43: Install app "httpbin" in namespace "foo" gatewayapi_test.go:43: Wait for app foo/httpbin to be ready gatewayapi_test.go:45: gatewayapi_test.go:45: STEP 2: Deploy the Gateway SMCP gatewayapi_test.go:78: gatewayapi_test.go:78: STEP 3: Deploy the Gateway API configuration including a single exposed route (i.e., /get) gatewayapi_test.go:84: gatewayapi_test.go:84: STEP 4: Wait for Gateway to be ready gatewayapi_test.go:85: Wait for condition condition=Ready on Gateway foo/gateway... gatewayapi_test.go:85: SUCCESS: Condition condition=Ready met by Gateway foo/gateway gatewayapi_test.go:87: gatewayapi_test.go:87: STEP 5: Verfiy the GatewayApi access the httpbin service using curl gatewayapi_test.go:89: FAILURE: Command failed: kubectl exec -n istio-system istio-ingressgateway-59d64db9d9-89d74 -c istio-proxy -- curl http://gateway.foo.svc.cluster.local:8080/get -H Host:httpbin.example.com -s -o /dev/null -w %{http_code} 000command terminated with exit code 7 error: exit status 7 gatewayapi_test.go:88: --- Attempt 1/60 failed. Retrying... gatewayapi_test.go:89: SUCCESS: Access the httpbin service with GatewayApi gatewayapi_test.go:88: --- Attempt 2/60 successful; total time: 1.96s subtest.go:23: subtest.go:28: Subtest completed in 11.46s (excluding cleanup) gatewayapi_test.go:80: gatewayapi_test.go:80: Performing cleanup gatewayapi_test.go:80: Cleanup completed in 0.20s gatewayapi_test.go:71: gatewayapi_test.go:71: Performing cleanup gatewayapi_test.go:71: Cleanup completed in 0.17s gatewayapi_test.go:37: gatewayapi_test.go:37: Performing cleanup gatewayapi_test.go:38: Deleting namespaces: [istio-system] gatewayapi_test.go:38: Creating namespaces: [istio-system] gatewayapi_test.go:39: Deleting namespaces: [foo] gatewayapi_test.go:39: Creating namespaces: [foo] gatewayapi_test.go:37: Cleanup completed in 95.04s --- PASS: TestGatewayApi/Deploy_the_Kubernetes_Gateway_API (106.87s) === RUN TestGatewayApi/Deploy_the_Gateway-Controller_Profile gatewayapi_test.go:109: gatewayapi_test.go:109: STEP 1: Install httpbin gatewayapi_test.go:110: Install app "httpbin" in namespace "foo" gatewayapi_test.go:110: Wait for app foo/httpbin to be ready gatewayapi_test.go:112: gatewayapi_test.go:112: STEP 2: Deploy SMCP with the profile gatewayapi_test.go:117: Wait for condition condition=Ready on smcp istio-system/basic... gatewayapi_test.go:117: SUCCESS: Condition condition=Ready met by smcp istio-system/basic gatewayapi_test.go:119: gatewayapi_test.go:119: STEP 3: delete default SMMR and create custom SMMR gatewayapi_test.go:124: gatewayapi_test.go:124: STEP 4: Deploy the Gateway API configuration including a single exposed route (i.e., /get) gatewayapi_test.go:130: gatewayapi_test.go:130: STEP 5: Wait for Gateway to be ready gatewayapi_test.go:131: Wait for condition condition=Ready on Gateway foo/gateway... gatewayapi_test.go:131: SUCCESS: Condition condition=Ready met by Gateway foo/gateway gatewayapi_test.go:133: gatewayapi_test.go:133: STEP 6: Verify the Gateway-Controller Profile access the httpbin service using curl gatewayapi_test.go:135: SUCCESS: Access the httpbin service with GatewayApi subtest.go:23: subtest.go:28: Subtest completed in 12.42s (excluding cleanup) gatewayapi_test.go:126: gatewayapi_test.go:126: Performing cleanup gatewayapi_test.go:126: Cleanup completed in 0.16s gatewayapi_test.go:104: gatewayapi_test.go:104: Performing cleanup gatewayapi_test.go:105: Deleting namespaces: [istio-system] gatewayapi_test.go:105: Creating namespaces: [istio-system] gatewayapi_test.go:106: Deleting namespaces: [foo] gatewayapi_test.go:106: Creating namespaces: [foo] gatewayapi_test.go:104: Cleanup completed in 40.96s --- PASS: TestGatewayApi/Deploy_the_Gateway-Controller_Profile (53.55s) test.go:77: test.go:82: Test completed in 171.66s (excluding cleanup) --- PASS: TestGatewayApi (171.66s) === RUN TestIngressGateways ingress_gateways_test.go:42: ingress_gateways_test.go:42: STEP 1: Apply default SMCP and SMMR manifests ingress_gateways_test.go:42: Wait for condition condition=Ready on smcp istio-system/basic... ingress_gateways_test.go:42: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 ingress_gateways_test.go:42: Wait for condition condition=Ready on smcp istio-system/basic... ingress_gateways_test.go:42: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 ingress_gateways_test.go:42: Wait for condition condition=Ready on smcp istio-system/basic... ingress_gateways_test.go:42: SUCCESS: Condition condition=Ready met by smcp istio-system/basic ingress_gateways_test.go:44: ingress_gateways_test.go:44: STEP 2: Install httpbin ingress_gateways_test.go:45: Install app "httpbin" in namespace "bookinfo" ingress_gateways_test.go:45: Wait for app bookinfo/httpbin to be ready === RUN TestIngressGateways/TrafficManagement_ingress_status_200_test ingress_gateways_test.go:50: ingress_gateways_test.go:50: STEP 1: Create httpbin Gateway and VirtualService with host set to httpbin.example.com ingress_gateways_test.go:57: ingress_gateways_test.go:57: STEP 2: Check if httpbin service is reachable through istio-ingressgateway ingress_gateways_test.go:59: FAILURE: expected status code 200 but got 503 Service Unavailable ingress_gateways_test.go:58: --- Attempt 1/60 failed. Retrying... ingress_gateways_test.go:59: FAILURE: expected status code 200 but got 503 Service Unavailable ingress_gateways_test.go:58: --- Attempt 2/60 failed. Retrying... ingress_gateways_test.go:59: FAILURE: expected status code 200 but got 503 Service Unavailable ingress_gateways_test.go:58: --- Attempt 3/60 failed. Retrying... ingress_gateways_test.go:59: SUCCESS: received expected status code 200 ingress_gateways_test.go:58: --- Attempt 4/60 successful; total time: 3.15s subtest.go:23: subtest.go:28: Subtest completed in 3.59s (excluding cleanup) --- PASS: TestIngressGateways/TrafficManagement_ingress_status_200_test (3.59s) === RUN TestIngressGateways/TrafficManagement_ingress_headers_test ingress_gateways_test.go:67: ingress_gateways_test.go:67: STEP 1: Create httpbin Gateway and VirtualService with host set to * ingress_gateways_test.go:70: ingress_gateways_test.go:70: STEP 2: Check if httpbin service is reachable through istio-ingressgateway ingress_gateways_test.go:72: FAILURE: expected status code 200 but got 404 Not Found ingress_gateways_test.go:71: --- Attempt 1/60 failed. Retrying... ingress_gateways_test.go:72: SUCCESS: received expected status code 200 ingress_gateways_test.go:71: --- Attempt 2/60 successful; total time: 1.04s subtest.go:23: subtest.go:28: Subtest completed in 1.48s (excluding cleanup) --- PASS: TestIngressGateways/TrafficManagement_ingress_headers_test (1.48s) test.go:77: test.go:82: Test completed in 32.24s (excluding cleanup) ingress_gateways_test.go:38: ingress_gateways_test.go:38: Performing cleanup ingress_gateways_test.go:39: Deleting namespaces: [bookinfo] ingress_gateways_test.go:39: Creating namespaces: [bookinfo] ingress_gateways_test.go:38: Cleanup completed in 18.81s --- PASS: TestIngressGateways (51.05s) === RUN TestIngressWithoutTlsTermination ingress_without_tls_termination_test.go:36: This test validates configuring an Gateway with TLS PassThrough ingress_without_tls_termination_test.go:37: Doc reference: https://istio.io/v1.14/docs/tasks/traffic-management/ingress/ingress-sni-passthrough/ ingress_without_tls_termination_test.go:46: ingress_without_tls_termination_test.go:46: STEP 1: Apply default SMCP and SMMR manifests ingress_without_tls_termination_test.go:46: Wait for condition condition=Ready on smcp istio-system/basic... ingress_without_tls_termination_test.go:46: SUCCESS: Condition condition=Ready met by smcp istio-system/basic ingress_without_tls_termination_test.go:48: ingress_without_tls_termination_test.go:48: STEP 2: Create NGINX Deployment ingress_without_tls_termination_test.go:49: Install app "nginx" in namespace "bookinfo" ingress_without_tls_termination_test.go:49: Wait for app bookinfo/nginx to be ready ingress_without_tls_termination_test.go:51: ingress_without_tls_termination_test.go:51: STEP 3: Verify NGINX server is running by connecting to it via loopback ingress_without_tls_termination_test.go:53: SUCCESS: Got expected Welcome to nginx response ingress_without_tls_termination_test.go:63: ingress_without_tls_termination_test.go:63: STEP 4: Configure Gateway resource with TLS passthrough for host nginx.example.com ingress_without_tls_termination_test.go:70: ingress_without_tls_termination_test.go:70: STEP 5: Verify NGINX is reachable from outside the cluster through the ingressgateway ingress_without_tls_termination_test.go:74: SUCCESS: string 'Welcome to nginx' found in response test.go:77: test.go:82: Test completed in 17.55s (excluding cleanup) ingress_without_tls_termination_test.go:41: ingress_without_tls_termination_test.go:41: Performing cleanup ingress_without_tls_termination_test.go:41: Cleanup completed in 0.77s --- PASS: TestIngressWithoutTlsTermination (18.32s) === RUN TestSecureGateways secure_gateways_test.go:59: secure_gateways_test.go:59: STEP 1: Apply default SMCP and SMMR manifests secure_gateways_test.go:59: Wait for condition condition=Ready on smcp istio-system/basic... secure_gateways_test.go:59: SUCCESS: Condition condition=Ready met by smcp istio-system/basic secure_gateways_test.go:61: secure_gateways_test.go:61: STEP 2: Install httpbin secure_gateways_test.go:62: Install app "httpbin" in namespace "bookinfo" secure_gateways_test.go:62: Wait for app bookinfo/httpbin to be ready secure_gateways_test.go:66: secure_gateways_test.go:66: STEP 3: Create TLS secrets === RUN TestSecureGateways/tls_single_host secure_gateways_test.go:76: secure_gateways_test.go:76: STEP 1: Configure a TLS ingress gateway for a single host secure_gateways_test.go:84: SUCCESS: string '-=[ teapot ]=-' found in response subtest.go:23: subtest.go:28: Subtest completed in 1.52s (excluding cleanup) --- PASS: TestSecureGateways/tls_single_host (1.52s) === RUN TestSecureGateways/tls_multiple_hosts secure_gateways_test.go:93: secure_gateways_test.go:93: STEP 1: configure Gateway with multiple TLS hosts secure_gateways_test.go:101: secure_gateways_test.go:101: STEP 2: check if helloworld-v1 responds with 200 OK secure_gateways_test.go:103: failed to get HTTP Response: Get "https://helloworld-v1.example.com:443/hello": tls: failed to verify certificate: x509: certificate is valid for *.apps.ci-op-11b996y7-8c5aa.cspilp.interop.ccitredhat.com, not helloworld-v1.example.com secure_gateways_test.go:103: FAILURE: response is nil; the HTTP request must have failed secure_gateways_test.go:102: --- Attempt 1/60 failed. Retrying... secure_gateways_test.go:103: failed to get HTTP Response: Get "https://helloworld-v1.example.com:443/hello": tls: failed to verify certificate: x509: certificate is valid for *.apps.ci-op-11b996y7-8c5aa.cspilp.interop.ccitredhat.com, not helloworld-v1.example.com secure_gateways_test.go:103: FAILURE: response is nil; the HTTP request must have failed secure_gateways_test.go:102: --- Attempt 2/60 failed. Retrying... secure_gateways_test.go:103: failed to get HTTP Response: Get "https://helloworld-v1.example.com:443/hello": tls: failed to verify certificate: x509: certificate is valid for *.apps.ci-op-11b996y7-8c5aa.cspilp.interop.ccitredhat.com, not helloworld-v1.example.com secure_gateways_test.go:103: FAILURE: response is nil; the HTTP request must have failed secure_gateways_test.go:102: --- Attempt 3/60 failed. Retrying... secure_gateways_test.go:103: failed to get HTTP Response: Get "https://helloworld-v1.example.com:443/hello": tls: failed to verify certificate: x509: certificate is valid for *.apps.ci-op-11b996y7-8c5aa.cspilp.interop.ccitredhat.com, not helloworld-v1.example.com secure_gateways_test.go:103: FAILURE: response is nil; the HTTP request must have failed secure_gateways_test.go:102: --- Attempt 4/60 failed. Retrying... secure_gateways_test.go:103: SUCCESS: received expected status code 200 secure_gateways_test.go:102: --- Attempt 5/60 successful; total time: 4.26s secure_gateways_test.go:109: secure_gateways_test.go:109: STEP 3: check if httpbin responds with teapot secure_gateways_test.go:111: SUCCESS: string '-=[ teapot ]=-' found in response subtest.go:23: subtest.go:28: Subtest completed in 4.75s (excluding cleanup) --- PASS: TestSecureGateways/tls_multiple_hosts (4.75s) === RUN TestSecureGateways/mutual_tls secure_gateways_test.go:119: secure_gateways_test.go:119: STEP 1: configure Gateway with tls.mode=Mutual secure_gateways_test.go:130: secure_gateways_test.go:130: STEP 2: check if SSL handshake fails when no client certificate is given secure_gateways_test.go:132: FAILURE: expected request to fail because no client certificate was provided; expected request to fail, but it succeeded with the following status: 418 Unknown secure_gateways_test.go:131: --- Attempt 1/60 failed. Retrying... secure_gateways_test.go:132: failed to get HTTP Response: Get "https://httpbin.example.com:443/status/418": remote error: tls: certificate required secure_gateways_test.go:132: SUCCESS: request failed as expected secure_gateways_test.go:131: --- Attempt 2/60 successful; total time: 1.08s secure_gateways_test.go:140: secure_gateways_test.go:140: STEP 3: check if SSL handshake succeeds when client certificate is given secure_gateways_test.go:142: SUCCESS: string '-=[ teapot ]=-' found in response subtest.go:23: subtest.go:28: Subtest completed in 1.77s (excluding cleanup) --- PASS: TestSecureGateways/mutual_tls (1.77s) test.go:77: test.go:82: Test completed in 27.09s (excluding cleanup) secure_gateways_test.go:53: secure_gateways_test.go:53: Performing cleanup secure_gateways_test.go:56: Deleting namespaces: [bookinfo] secure_gateways_test.go:56: Creating namespaces: [bookinfo] secure_gateways_test.go:53: Cleanup completed in 18.13s --- PASS: TestSecureGateways (45.22s) PASS ok github.com/maistra/maistra-test-tool/pkg/tests/tasks/traffic/ingress 286.679s DONE 108 tests, 20 skipped, 2 failures in 2063.880s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestPluginCaCert install.go:32: cert-manager-operator is not installed, starting installation install.go:59: install.go:59: STEP 1: Create namespace for cert-manager-operator install.go:60: Creating namespaces: [cert-manager-operator] install.go:62: install.go:62: STEP 2: Install cert-manager-operator install.go:67: Waiting for cert-manager-operator to succeed install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 1/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 2/60 failed. Retrying... install.go:69: --- Attempt 3/60 successful; total time: 2.44s install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: SUCCESS: Condition jsonpath='{.status.phase}'=Succeeded met by csv cert-manager-operator/cert-manager-operator.v1.11.1 install.go:76: Pod cert-manager-operator-controller-manager-c5c6bddcf-z6tzf in namespace cert-manager-operator is ready! install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 1/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 2/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 3/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 4/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 5/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 6/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 7/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 8/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 9/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 10/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 11/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 12/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 13/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 14/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 15/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 16/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 17/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 18/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 19/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 20/60 failed. Retrying in 4s... install.go:77: FAILURE: more than one pod found using selector "app=cert-manager" in namespace cert-manager: [{cert-manager cert-manager-5c5bc4f8f4-kkvrj} {cert-manager cert-manager-84b449cdff-r4sf8}] install.go:77: --- Attempt 21/60 failed. Retrying in 4s... install.go:77: Pod cert-manager-5c5bc4f8f4-kkvrj in namespace cert-manager is ready! install.go:77: --- Attempt 22/60 successful; total time: 87.12s install.go:41: install.go:41: STEP 3: Create root ca plugin_ca_test.go:58: plugin_ca_test.go:58: STEP 4: Uninstall existing SMCP plugin_ca_test.go:59: Deleting namespaces: [istio-system] plugin_ca_test.go:59: Creating namespaces: [istio-system] plugin_ca_test.go:61: plugin_ca_test.go:61: STEP 5: Create intermediate CA certificate for Istio plugin_ca_test.go:64: plugin_ca_test.go:64: STEP 6: Deploy SMCP v2.4 and SMMR plugin_ca_test.go:66: Wait for condition condition=Ready on smcp istio-system/basic... plugin_ca_test.go:66: SUCCESS: Condition condition=Ready met by smcp istio-system/basic plugin_ca_test.go:68: plugin_ca_test.go:68: STEP 7: Verify that cacerts secret was detected plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 1/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 2/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 3/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 4/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 5/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 6/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 7/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 8/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 9/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 10/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 11/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 12/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 13/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 14/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 15/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 16/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 17/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 18/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 19/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 20/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 21/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 22/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 23/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 24/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 25/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 26/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 27/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 28/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 29/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 30/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 31/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 32/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 33/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 34/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 35/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 36/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 37/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 38/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 39/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 40/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 41/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 42/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 43/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 44/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 45/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 46/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 47/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 48/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 49/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 50/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 51/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 52/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 53/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 54/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 55/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 56/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 57/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 58/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 59/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found; full output: 2023-08-02T17:52:08.955480Z info FLAG: --caCertFile="" 2023-08-02T17:52:08.955504Z info FLAG: --clusterAliases="[]" 2023-08-02T17:52:08.955508Z info FLAG: --clusterID="Kubernetes" 2023-08-02T17:52:08.955510Z info FLAG: --clusterRegistriesNamespace="istio-system" 2023-08-02T17:52:08.955513Z info FLAG: --configDir="" 2023-08-02T17:52:08.955515Z info FLAG: --ctrlz_address="localhost" 2023-08-02T17:52:08.955518Z info FLAG: --ctrlz_port="9876" 2023-08-02T17:52:08.955521Z info FLAG: --domain="cluster.local" 2023-08-02T17:52:08.955525Z info FLAG: --enableCRDScan="false" 2023-08-02T17:52:08.955527Z info FLAG: --enableIngressClassName="false" 2023-08-02T17:52:08.955530Z info FLAG: --enableNodeAccess="false" 2023-08-02T17:52:08.955533Z info FLAG: --federationAddr=":8188" 2023-08-02T17:52:08.955535Z info FLAG: --grpcAddr=":15010" 2023-08-02T17:52:08.955538Z info FLAG: --help="false" 2023-08-02T17:52:08.955540Z info FLAG: --httpAddr=":8080" 2023-08-02T17:52:08.955543Z info FLAG: --httpsAddr=":15017" 2023-08-02T17:52:08.955547Z info FLAG: --keepaliveInterval="30s" 2023-08-02T17:52:08.955550Z info FLAG: --keepaliveMaxServerConnectionAge="30m0s" 2023-08-02T17:52:08.955553Z info FLAG: --keepaliveTimeout="10s" 2023-08-02T17:52:08.955555Z info FLAG: --kubeconfig="" 2023-08-02T17:52:08.955558Z info FLAG: --kubernetesApiBurst="160" 2023-08-02T17:52:08.955562Z info FLAG: --kubernetesApiQPS="80" 2023-08-02T17:52:08.955565Z info FLAG: --logKubernetesApiRequests="false" 2023-08-02T17:52:08.955568Z info FLAG: --log_as_json="false" 2023-08-02T17:52:08.955570Z info FLAG: --log_caller="" 2023-08-02T17:52:08.955573Z info FLAG: --log_output_level="default:info" 2023-08-02T17:52:08.955576Z info FLAG: --log_rotate="" 2023-08-02T17:52:08.955579Z info FLAG: --log_rotate_max_age="30" 2023-08-02T17:52:08.955581Z info FLAG: --log_rotate_max_backups="1000" 2023-08-02T17:52:08.955584Z info FLAG: --log_rotate_max_size="104857600" 2023-08-02T17:52:08.955587Z info FLAG: --log_stacktrace_level="default:none" 2023-08-02T17:52:08.955594Z info FLAG: --log_target="[stdout]" 2023-08-02T17:52:08.955596Z info FLAG: --memberRollName="default" 2023-08-02T17:52:08.955599Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2023-08-02T17:52:08.955602Z info FLAG: --monitoringAddr=":15014" 2023-08-02T17:52:08.955604Z info FLAG: --namespace="istio-system" 2023-08-02T17:52:08.955607Z info FLAG: --networksConfig="./etc/istio/config/meshNetworks" 2023-08-02T17:52:08.955609Z info FLAG: --profile="true" 2023-08-02T17:52:08.955614Z info FLAG: --registries="[Kubernetes]" 2023-08-02T17:52:08.955617Z info FLAG: --resync="1m0s" 2023-08-02T17:52:08.955619Z info FLAG: --secureGRPCAddr=":15012" 2023-08-02T17:52:08.955622Z info FLAG: --shutdownDuration="10s" 2023-08-02T17:52:08.955625Z info FLAG: --tls-cipher-suites="[]" 2023-08-02T17:52:08.955628Z info FLAG: --tlsCertFile="" 2023-08-02T17:52:08.955630Z info FLAG: --tlsKeyFile="" 2023-08-02T17:52:08.955633Z info FLAG: --vklog="0" 2023-08-02T17:52:08.964726Z info initializing Istiod admin server 2023-08-02T17:52:08.965537Z info klog Config not found: /var/run/secrets/remote/config 2023-08-02T17:52:08.966448Z info starting HTTP service at [::]:8080 2023-08-02T17:52:08.966954Z info smmr Member namespace list updated: [] 2023-08-02T17:52:08.977520Z info initializing mesh configuration ./etc/istio/config/mesh 2023-08-02T17:52:08.984430Z info controllers starting controller=configmap istio-basic 2023-08-02T17:52:08.984642Z info Loaded MeshNetworks config from Kubernetes API server. 2023-08-02T17:52:08.984658Z info mesh networks configuration updated to: { } 2023-08-02T17:52:08.985812Z info Loaded MeshConfig config from Kubernetes API server. 2023-08-02T17:52:08.986210Z info mesh configuration updated to: { "proxyListenPort": 15001, "connectTimeout": "10s", "protocolDetectionTimeout": "0s", "ingressClass": "istio", "ingressService": "istio-ingressgateway", "ingressControllerMode": "OFF", "enableTracing": true, "defaultConfig": { "configPath": "./etc/istio/proxy", "binaryPath": "/usr/local/bin/envoy", "serviceCluster": "istio-proxy", "drainDuration": "45s", "parentShutdownDuration": "60s", "discoveryAddress": "istiod-basic.istio-system.svc:15012", "proxyAdminPort": 15000, "controlPlaneAuthPolicy": "MUTUAL_TLS", "statNameLength": 189, "concurrency": 2, "tracing": { "zipkin": { "address": "zipkin.istio-system:9411" } }, "proxyMetadata": { "ISTIO_META_DNS_AUTO_ALLOCATE": "true", "ISTIO_META_DNS_CAPTURE": "true", "PROXY_XDS_VIA_AGENT": "true" }, "statusPort": 15020, "terminationDrainDuration": "5s" }, "outboundTrafficPolicy": { "mode": "ALLOW_ANY" }, "enableAutoMtls": true, "trustDomain": "cluster.local", "defaultServiceExportTo": [ "*" ], "defaultVirtualServiceExportTo": [ "*" ], "defaultDestinationRuleExportTo": [ "*" ], "rootNamespace": "istio-system", "localityLbSetting": { "enabled": true }, "dnsRefreshRate": "300s", "thriftConfig": { }, "enablePrometheusMerge": true, "extensionProviders": [ { "name": "prometheus", "prometheus": { } }, { "name": "stackdriver", "stackdriver": { } }, { "name": "envoy", "envoyFileAccessLog": { "path": "/dev/stdout" } } ], "defaultProviders": { } } 2023-08-02T17:52:08.992546Z info initializing mesh networks from mesh config watcher 2023-08-02T17:52:08.992641Z info mesh configuration: { "proxyListenPort": 15001, "connectTimeout": "10s", "protocolDetectionTimeout": "0s", "ingressClass": "istio", "ingressService": "istio-ingressgateway", "ingressControllerMode": "OFF", "enableTracing": true, "defaultConfig": { "configPath": "./etc/istio/proxy", "binaryPath": "/usr/local/bin/envoy", "serviceCluster": "istio-proxy", "drainDuration": "45s", "parentShutdownDuration": "60s", "discoveryAddress": "istiod-basic.istio-system.svc:15012", "proxyAdminPort": 15000, "controlPlaneAuthPolicy": "MUTUAL_TLS", "statNameLength": 189, "concurrency": 2, "tracing": { "zipkin": { "address": "zipkin.istio-system:9411" } }, "proxyMetadata": { "ISTIO_META_DNS_AUTO_ALLOCATE": "true", "ISTIO_META_DNS_CAPTURE": "true", "PROXY_XDS_VIA_AGENT": "true" }, "statusPort": 15020, "terminationDrainDuration": "5s" }, "outboundTrafficPolicy": { "mode": "ALLOW_ANY" }, "enableAutoMtls": true, "trustDomain": "cluster.local", "defaultServiceExportTo": [ "*" ], "defaultVirtualServiceExportTo": [ "*" ], "defaultDestinationRuleExportTo": [ "*" ], "rootNamespace": "istio-system", "localityLbSetting": { "enabled": true }, "dnsRefreshRate": "300s", "thriftConfig": { }, "enablePrometheusMerge": true, "extensionProviders": [ { "name": "prometheus", "prometheus": { } }, { "name": "stackdriver", "stackdriver": { } }, { "name": "envoy", "envoyFileAccessLog": { "path": "/dev/stdout" } } ], "defaultProviders": { } } 2023-08-02T17:52:08.992651Z info version: OSSM_2.4.1-1-466b6910fb9c61ad1d30759b511e1c31b7cf7dd3-Clean 2023-08-02T17:52:08.992830Z info flags: { "ServerOptions": { "HTTPAddr": ":8080", "HTTPSAddr": ":15017", "GRPCAddr": ":15010", "MonitoringAddr": ":15014", "EnableProfiling": true, "TLSOptions": { "CaCertFile": "", "CertFile": "", "KeyFile": "", "TLSCipherSuites": null, "CipherSuits": null }, "SecureGRPCAddr": ":15012", "FederationAddr": ":8188" }, "InjectionOptions": { "InjectionDirectory": "./var/lib/istio/inject" }, "PodName": "istiod-basic-6dbb5fcd4d-g82lq", "Namespace": "istio-system", "Revision": "basic", "MeshConfigFile": "./etc/istio/config/mesh", "Network": "", "NetworksConfigFile": "./etc/istio/config/meshNetworks", "RegistryOptions": { "FileDir": "", "Registries": [ "Kubernetes" ], "KubeOptions": { "SystemNamespace": "", "MeshServiceController": null, "ResyncPeriod": 60000000000, "DomainSuffix": "cluster.local", "MemberRollName": "default", "ClusterID": "Kubernetes", "ClusterAliases": {}, "Metrics": null, "XDSUpdater": null, "NetworksWatcher": null, "MeshWatcher": null, "EndpointMode": 1, "KubernetesAPIQPS": 80, "KubernetesAPIBurst": 160, "LogKubernetesAPIRequests": false, "SyncTimeout": 0, "DiscoveryNamespacesFilter": null, "EnableCRDScan": false, "EnableNodeAccess": false, "EnableIngressClassName": false }, "ClusterRegistriesNamespace": "istio-system", "KubeConfig": "", "DistributionCacheRetention": 60000000000, "DistributionTrackingEnabled": false }, "CtrlZOptions": { "Port": 9876, "Address": "localhost" }, "KeepaliveOptions": { "Time": 30000000000, "Timeout": 10000000000, "MaxServerConnectionAge": 1800000000000, "MaxServerConnectionAgeGrace": 10000000000 }, "ShutdownDuration": 10000000000, "JwtRule": "" } 2023-08-02T17:52:08.992841Z info initializing mesh handlers 2023-08-02T17:52:08.992873Z info model reloading network gateways 2023-08-02T17:52:08.992906Z info creating CA and initializing public key 2023-08-02T17:52:08.992944Z info Using istiod file format for signing ca files 2023-08-02T17:52:08.992959Z info Use self-signed certificate as the CA certificate 2023-08-02T17:52:08.996460Z info pkica Failed to get secret (error: secrets "istio-ca-secret" not found), will create one 2023-08-02T17:52:09.244488Z info pkica Using self-generated public key: -----BEGIN CERTIFICATE----- MIIC/TCCAeWgAwIBAgIRAJW/d5Dfxyi16KdUj1aCa2owDQYJKoZIhvcNAQELBQAw GDEWMBQGA1UEChMNY2x1c3Rlci5sb2NhbDAeFw0yMzA4MDIxNzUyMDlaFw0zMzA3 MzAxNzUyMDlaMBgxFjAUBgNVBAoTDWNsdXN0ZXIubG9jYWwwggEiMA0GCSqGSIb3 DQEBAQUAA4IBDwAwggEKAoIBAQCwo3cT0ohZYRUl/yV76gBNnUzsbA+DhUXIBcEN OiHIYpX3wfc/beVLaGS+0hqBdINXWO/O6qsN54T+96jfdJ/QnvidUVNdiqIQYXlQ srHFMH1W28o7mn8nD2t62sr5+CSCjTYROnFGDjslEZ/IOBkTfwkHKHoGHCwnUKuk rAaKAJWtbPBX152EmeeNi0sQ+bEPo/CoNjJnFeT0oAq9FvnVUGE/E1ku2gl9Vw4s JlxMdmXn7Sj+tj2gstPUJxs9doNamTJP8VhhpQIeZjiIC2U0vZpP41+sRixyWq0I FzpNkC/kkOiTrK+xJjJng+mv+RjRRfhixsEzYr53RrMFnvibAgMBAAGjQjBAMA4G A1UdDwEB/wQEAwICBDAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBT2iekZQWiQ n3/Jj1KdLjJ6e1zXbzANBgkqhkiG9w0BAQsFAAOCAQEAOxAlVjwfH38cZShJzPfO HRZIfFCt3Ac02QIqV+Kbmt7LeKs2/EKWJ57iovNJjuHHBdm1Eyrlp6prkWju6mnN oTQ0ANjoELwKK933gQbXQQ0+btCyiPsMHWVa5UMFZXMMLGi8bHa2Qs2acmc6EENY ZaeoJzQXW3ZwbIWDrkZzSKOjddSyGpJUQgrLw7HxJlykul2rxre7XMqNWt7rLOAy KiXAKSSCoqzM9pQzMe2vimhIrMnEQFpElAlr8SkrGzZKiV1aAibclPT+4BPo341o ZMl29eCuYTjMhmyu7WyREJ6n6/WRohxCKs/H4glMPBmf4/VFwBZ+adr5lZXqMNAG nw== -----END CERTIFICATE----- 2023-08-02T17:52:09.244522Z info rootcertrotator Set up back off time 59m39s to start rotator. 2023-08-02T17:52:09.244596Z info initializing Istiod DNS certificates host: istiod-basic.istio-system.svc, custom host: 2023-08-02T17:52:09.244838Z info rootcertrotator Jitter is enabled, wait 59m39s before starting root cert rotator. 2023-08-02T17:52:09.307372Z info Generating istiod-signed cert for [istiod-basic.istio-system.svc istiod.istio-system.svc istiod-remote.istio-system.svc istio-pilot.istio-system.svc]: -----BEGIN CERTIFICATE----- MIIDiTCCAnGgAwIBAgIQG3RH7C5iicMZ2WGHiEtdXDANBgkqhkiG9w0BAQsFADAY MRYwFAYDVQQKEw1jbHVzdGVyLmxvY2FsMB4XDTIzMDgwMjE3NTAwOVoXDTMzMDcz MDE3NTIwOVowADCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKDWJm06 W5McTw/kHLTK2MljNzVXNAbQTgvqEMQ8VyUaIldqRCm0OeR6R782esFhziQiLAnl S+HpKVqtQo4LsqP3vHcuCwrrihqQSR6GPceE1TkFwbj6BIFfsDyrL1u9x/ZVpPFS 3P8M5kq/sAVNp+sm/Z35FI1SRuh/tGIVYXEWjaf1hAQsu82goe6JrqNtSED1Y5yZ d9Qs2NnBA3H9XyiTlmiCOac3g8sCGnX6DsneoK/YBO7tOmIU4tRoum3K8IQTs+fa bgvgNVwsBayoKxmo/BavAqiuLd02HcnbuQNMjED7ZHv/ifRskVq1w5qP3x+HDh3Y DFR0yXqNOppbqRMCAwEAAaOB5jCB4zAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYw FAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAU 9onpGUFokJ9/yY9SnS4yentc128wgYIGA1UdEQEB/wR4MHaCHWlzdGlvZC1iYXNp Yy5pc3Rpby1zeXN0ZW0uc3Zjghdpc3Rpb2QuaXN0aW8tc3lzdGVtLnN2Y4IeaXN0 aW9kLXJlbW90ZS5pc3Rpby1zeXN0ZW0uc3Zjghxpc3Rpby1waWxvdC5pc3Rpby1z eXN0ZW0uc3ZjMA0GCSqGSIb3DQEBCwUAA4IBAQA52eLnViZSkRrCKQiqKZU/Uw+a JDTBFEugZkaSxi4j0GB+8L9jOpDtw8fB7VES8vihlmWbjibqJn3m8TjvCB222fN2 Z3Lare1graUMcuLr5Bg33lMG2IgkAcCpcrSDuxEAvuGA9VCx9CErawUHyl16uuKB zU93MmBExOzEZMybHh+CJRq5Bgl71vKfN4u96V8miDPRCIbf8/l/oCIDL+7pcCT+ baEd0bWrhBnigkg7csjV1gP5HQOveJdIH3owOv5z+NrAkTassrZYNvhbANWDniC/ hkcm5eP+I3bvcvFhX+TpKcTBEq1tkR5hFx9ZAq7RaXWIGIdz7DK32GD2QBxL -----END CERTIFICATE----- 2023-08-02T17:52:09.307412Z info Using istiod file format for signing ca files 2023-08-02T17:52:09.307422Z info No plugged-in cert at etc/cacerts/ca-key.pem; self-signed cert is used 2023-08-02T17:52:09.307529Z info x509 cert - Issuer: "O=cluster.local", Subject: "", SN: 1b7447ec2e6289c319d96187884b5d5c, NotBefore: "2023-08-02T17:50:09Z", NotAfter: "2033-07-30T17:52:09Z" 2023-08-02T17:52:09.307539Z info Istiod certificates are reloaded 2023-08-02T17:52:09.307545Z info initializing controllers 2023-08-02T17:52:09.307597Z info spiffe Added 1 certs to trust domain cluster.local in peer cert verifier 2023-08-02T17:52:09.307608Z info Go Cipher suites are [] 2023-08-02T17:52:09.307612Z info OpenSSL Cipher suites are [] 2023-08-02T17:52:09.307616Z info Go ECDH Curves are [] 2023-08-02T17:52:09.307619Z info OpenSSL ECDH Curves are [] 2023-08-02T17:52:09.307916Z warn federation ResyncPeriod not specified, defaulting to 1m0s component=federation-discovery-controller 2023-08-02T17:52:09.307963Z info No certificates specified, skipping K8S DNS certificate controller 2023-08-02T17:52:09.308232Z info Adding Kubernetes registry adapter 2023-08-02T17:52:09.308295Z info spiffe Added 1 certs to trust domain cluster.local in peer cert verifier 2023-08-02T17:52:09.308303Z info initializing secure discovery service 2023-08-02T17:52:09.308334Z info initializing secure webhook server for istiod webhooks 2023-08-02T17:52:09.316576Z info initializing sidecar injector 2023-08-02T17:52:09.329287Z info initializing config validator 2023-08-02T17:52:09.329301Z info initializing registry event handlers 2023-08-02T17:52:09.329319Z info starting discovery service 2023-08-02T17:52:09.331065Z info Starting Istiod Server with primary cluster Kubernetes 2023-08-02T17:52:09.331141Z info ControlZ available at 127.0.0.1:9876 2023-08-02T17:52:09.331213Z info kube Initializing Kubernetes service registry "Kubernetes" 2023-08-02T17:52:09.331237Z info federation Starting controller component=federation-discovery-controller 2023-08-02T17:52:09.331279Z info kube should join leader-election for cluster Kubernetes: false 2023-08-02T17:52:09.331309Z info initializing Kubernetes credential reader for cluster Kubernetes 2023-08-02T17:52:09.331322Z info federation Starting controller component=federation-exports-controller 2023-08-02T17:52:09.331287Z info klog attempting to acquire leader lease istio-system/servicemesh-federation... 2023-08-02T17:52:09.331321Z info federation Starting controller component=federation-imports-controller 2023-08-02T17:52:09.332312Z info Starting ADS server 2023-08-02T17:52:09.332391Z info Starting IstioD CA 2023-08-02T17:52:09.332416Z info JWT policy is first-party-jwt 2023-08-02T17:52:09.332550Z info Istiod CA has started 2023-08-02T17:52:09.332827Z info controllers starting controller=healthcheck 2023-08-02T17:52:09.333289Z info klog Waiting for caches to sync for smmr 2023-08-02T17:52:09.333431Z info controllers starting controller=unregister_workloadentry 2023-08-02T17:52:09.333523Z info kube Starting Pilot K8S CRD controller controller=crd-controller 2023-08-02T17:52:09.333550Z info kube controller "networking.istio.io/v1alpha3/EnvoyFilter" is syncing... controller=crd-controller 2023-08-02T17:52:09.333712Z info Starting multicluster remote secrets controller 2023-08-02T17:52:09.335801Z info kube controller "networking.istio.io/v1alpha3/EnvoyFilter" is syncing... controller=crd-controller 2023-08-02T17:52:09.338855Z info controllers starting controller=configmap istio-sidecar-injector-basic 2023-08-02T17:52:09.340355Z info kube controller "networking.istio.io/v1alpha3/VirtualService" is syncing... controller=crd-controller 2023-08-02T17:52:09.340360Z info multicluster remote secrets controller cache synced in 6.680753ms 2023-08-02T17:52:09.340397Z info controllers starting controller=multicluster secret 2023-08-02T17:52:09.346539Z info klog successfully acquired lease istio-system/servicemesh-federation 2023-08-02T17:52:09.346584Z info leader election lock obtained: servicemesh-federation 2023-08-02T17:52:09.348619Z info kube controller "networking.istio.io/v1alpha3/DestinationRule" is syncing... controller=crd-controller 2023-08-02T17:52:09.365451Z info kube controller "networking.istio.io/v1beta1/ProxyConfig" is syncing... controller=crd-controller 2023-08-02T17:52:09.397950Z info kube controller "networking.istio.io/v1alpha3/EnvoyFilter" is syncing... controller=crd-controller 2023-08-02T17:52:09.434374Z info klog Caches are synced for smmr 2023-08-02T17:52:09.462810Z info kube controller "networking.istio.io/v1alpha3/EnvoyFilter" is syncing... controller=crd-controller 2023-08-02T17:52:09.467917Z info smmr Cache synced for listener "dynamic-informers" 2023-08-02T17:52:09.467926Z info smmr Cache synced for listener "namespace logger" 2023-08-02T17:52:09.467941Z info smmr Cache synced for listener "istio-informers" 2023-08-02T17:52:09.467968Z info smmr Cache synced for listener "openshift-route-informers" 2023-08-02T17:52:09.467990Z info smmr Cache synced for listener "kubernetes-informers" 2023-08-02T17:52:09.467977Z info smmr Member namespace list updated: [foo istio-system] 2023-08-02T17:52:09.531767Z info smmr Cache synced for listener "MaistraDiscoveryNamespacesFilter" 2023-08-02T17:52:09.531952Z info federation Controller synced in 200.626481ms component=federation-exports-controller 2023-08-02T17:52:09.531990Z info federation Starting workers component=federation-exports-controller 2023-08-02T17:52:09.531964Z info federation Controller synced in 200.735324ms component=federation-discovery-controller 2023-08-02T17:52:09.532003Z info federation Starting workers component=federation-discovery-controller 2023-08-02T17:52:09.533187Z info federation Controller synced in 201.868272ms component=federation-imports-controller 2023-08-02T17:52:09.533221Z info federation Starting workers component=federation-imports-controller 2023-08-02T17:52:09.533256Z info Waiting for caches to be synced 2023-08-02T17:52:09.533287Z info kube joining leader-election for istio-namespace-controller-election in istio-system on cluster Kubernetes 2023-08-02T17:52:09.533344Z info klog attempting to acquire leader lease istio-system/istio-namespace-controller-election... 2023-08-02T17:52:09.542629Z info klog successfully acquired lease istio-system/istio-namespace-controller-election 2023-08-02T17:52:09.542681Z info leader election lock obtained: istio-namespace-controller-election 2023-08-02T17:52:09.542726Z info kube starting namespace controller for cluster Kubernetes 2023-08-02T17:52:09.542854Z info controllers starting controller=namespace controller 2023-08-02T17:52:09.542867Z info smmr Cache synced for listener "namespace-controller" 2023-08-02T17:52:09.549612Z error controllers error handling foo/istio-ca-root-cert, and retry budget exceeded: error when updating configmap istio-ca-root-cert: Operation cannot be fulfilled on configmaps "istio-ca-root-cert": the object has been modified; please apply your changes to the latest version and try again controller=namespace controller 2023-08-02T17:52:09.561996Z info ads Incremental push, service istiod-basic.istio-system.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2023-08-02T17:52:09.562016Z info kube kube controller for Kubernetes synced after 228.685698ms 2023-08-02T17:52:09.563090Z info kube Pilot K8S CRD controller synced in 229.624862ms controller=crd-controller 2023-08-02T17:52:09.565133Z info All controller caches have been synced up in 31.875076ms 2023-08-02T17:52:09.663322Z info ads Push debounce stable[1] 18 for config Secret/foo/builder-dockercfg-mjwts and 16 more configs: 100.220238ms since last change, 179.370149ms since last push, full=true 2023-08-02T17:52:09.663638Z info ads XDS: Pushing:2023-08-02T17:52:09Z/1 Services:1 ConnectedEndpoints:0 Version:2023-08-02T17:52:09Z/1 2023-08-02T17:52:09.692473Z info ads All caches have been synced up in 738.873102ms, marking server ready 2023-08-02T17:52:09.692568Z info starting secure gRPC discovery service at [::]:15012 2023-08-02T17:52:09.692621Z info starting gRPC discovery service at [::]:15010 2023-08-02T17:52:09.692653Z info starting webhook service at [::]:15017 2023-08-02T17:52:09.692628Z info federation starting federation service discovery at [::]:8188 component=federation-server 2023-08-02T17:52:10.035907Z info ads Full push, new service istio-system/istiod-basic.istio-system.svc.cluster.local 2023-08-02T17:52:10.136234Z info ads Push debounce stable[2] 1 for config ServiceEntry/istio-system/istiod-basic.istio-system.svc.cluster.local: 100.270839ms since last change, 100.270548ms since last push, full=true 2023-08-02T17:52:10.136590Z info ads XDS: Pushing:2023-08-02T17:52:10Z/2 Services:1 ConnectedEndpoints:0 Version:2023-08-02T17:52:10Z/2 2023-08-02T17:52:10.498400Z info ads Incremental push, service istio-ingressgateway.istio-system.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2023-08-02T17:52:10.537216Z info Sidecar injection request for istio-system/istio-ingressgateway-86497c6c97-***** (actual name not yet known) 2023-08-02T17:52:10.537239Z info Skipping istio-system/istio-ingressgateway-86497c6c97-***** (actual name not yet known) due to policy check 2023-08-02T17:52:10.598731Z info ads Push debounce stable[3] 10 for config DestinationRule/istio-system/default and 7 more configs: 100.223118ms since last change, 318.200148ms since last push, full=true 2023-08-02T17:52:10.600333Z info ads XDS: Pushing:2023-08-02T17:52:10Z/3 Services:2 ConnectedEndpoints:0 Version:2023-08-02T17:52:10Z/3 2023-08-02T17:52:11.274929Z warn Insecure first-party-jwt option used to validate token; use third-party-jwt 2023-08-02T17:52:11.281125Z info ads ADS: new connection for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system-1 2023-08-02T17:52:11.281936Z info ads CDS: PUSH request for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:9 size:8.4kB cached:0/8 2023-08-02T17:52:11.290123Z info ads EDS: PUSH request for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:8 size:1.0kB empty:3 cached:0/8 2023-08-02T17:52:11.417699Z warn Insecure first-party-jwt option used to validate token; use third-party-jwt 2023-08-02T17:52:11.428270Z info ads LDS: PUSH request for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:0 size:0B 2023-08-02T17:52:11.968653Z info http: TLS handshake error from 10.128.0.31:40590: remote error: tls: bad certificate 2023-08-02T17:52:12.071511Z info ads Incremental push, service istio-ingressgateway.istio-system.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2023-08-02T17:52:12.084496Z info ads CDS: PUSH for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:9 size:8.4kB cached:0/8 2023-08-02T17:52:12.084673Z info ads EDS: PUSH for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:8 size:1.0kB empty:3 cached:0/8 2023-08-02T17:52:12.084750Z info ads LDS: PUSH for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:0 size:0B 2023-08-02T17:52:12.100068Z info ads Full push, new service istio-system/istio-ingressgateway.istio-system.svc.cluster.local 2023-08-02T17:52:12.200868Z info ads Push debounce stable[4] 2 for config ServiceEntry/istio-system/istio-ingressgateway.istio-system.svc.cluster.local: 100.71405ms since last change, 129.301171ms since last push, full=true 2023-08-02T17:52:12.201129Z info ads XDS: Pushing:2023-08-02T17:52:12Z/4 Services:2 ConnectedEndpoints:1 Version:2023-08-02T17:52:12Z/4 2023-08-02T17:52:12.201403Z info ads CDS: PUSH for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:9 size:8.6kB cached:5/8 2023-08-02T17:52:12.201523Z info ads EDS: PUSH INC for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:3 size:622B empty:0 cached:0/3 2023-08-02T17:52:12.201568Z info ads LDS: PUSH for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:0 size:0B 2023-08-02T17:52:15.750323Z info ads Push debounce stable[5] 1 for config Secret/istio-system/cacerts-2r8sg: 100.855825ms since last change, 100.855554ms since last push, full=false 2023-08-02T17:52:15.750376Z info ads XDS: Incremental Pushing:2023-08-02T17:52:12Z/4 ConnectedEndpoints:1 Version:2023-08-02T17:52:12Z/4 2023-08-02T17:52:20.570295Z info http: TLS handshake error from 10.128.0.31:53156: remote error: tls: bad certificate 2023-08-02T17:52:20.658682Z info ads Push debounce stable[6] 2 for config Secret/istio-system/cacerts and 1 more configs: 100.317589ms since last change, 115.676559ms since last push, full=false 2023-08-02T17:52:20.658731Z info ads XDS: Incremental Pushing:2023-08-02T17:52:12Z/4 ConnectedEndpoints:1 Version:2023-08-02T17:52:12Z/4 2023-08-02T17:52:21.576235Z info http: TLS handshake error from 10.128.0.31:53158: remote error: tls: bad certificate 2023-08-02T17:52:23.582662Z info http: TLS handshake error from 10.128.0.31:53172: remote error: tls: bad certificate 2023-08-02T17:52:23.650448Z info http: TLS handshake error from 10.128.0.31:53180: remote error: tls: bad certificate 2023-08-02T17:52:24.655854Z info http: TLS handshake error from 10.128.0.31:53194: remote error: tls: bad certificate 2023-08-02T17:52:26.662109Z info http: TLS handshake error from 10.128.0.31:53200: remote error: tls: bad certificate 2023-08-02T17:52:26.840067Z info http: TLS handshake error from 10.128.0.31:53202: remote error: tls: bad certificate 2023-08-02T17:52:27.845300Z info http: TLS handshake error from 10.128.0.31:47942: remote error: tls: bad certificate 2023-08-02T17:52:29.851044Z info http: TLS handshake error from 10.128.0.31:47946: remote error: tls: bad certificate 2023-08-02T17:52:29.869172Z info http: TLS handshake error from 10.128.0.31:47948: remote error: tls: bad certificate 2023-08-02T17:52:30.874619Z info http: TLS handshake error from 10.128.0.31:47964: remote error: tls: bad certificate 2023-08-02T17:52:32.880338Z info http: TLS handshake error from 10.128.0.31:47978: remote error: tls: bad certificate plugin_ca_test.go:69: Last attempt (60/60) failed. plugin_ca_test.go:76: plugin_ca_test.go:76: STEP 8: Deploy httpbin and sleep plugin_ca_test.go:77: Install app "httpbin" in namespace "foo" plugin_ca_test.go:77: Install app "sleep" in namespace "foo" plugin_ca_test.go:77: Wait for app foo/httpbin to be ready plugin_ca_test.go:77: FATAL: Command failed: kubectl -n foo rollout status deploy/httpbin --timeout=4m0s Waiting for deployment "httpbin" rollout to finish: 0 out of 1 new replicas have been updated... error: timed out waiting for the condition error: exit status 1 test.go:77: test.go:79: Test failed in 436.34s (excluding cleanup) test.go:95: Capturing cluster state using must-gather registry.redhat.io/openshift-service-mesh/istio-must-gather-rhel8:2.4 test.go:99: /go/src/maistra-test-tool/tests/result-20230802171548/v2.4/failures-must-gather/20230802175731-TestPluginCaCert plugin_ca_test.go:48: plugin_ca_test.go:48: Performing cleanup plugin_ca_test.go:52: Deleting namespaces: [foo] plugin_ca_test.go:52: Creating namespaces: [foo] install.go:49: Deleting namespaces: [cert-manager-operator] install.go:50: Deleting namespaces: [cert-manager] plugin_ca_test.go:48: Cleanup completed in 26.82s --- FAIL: TestPluginCaCert (487.87s) FAIL FAIL github.com/maistra/maistra-test-tool/pkg/tests/tasks/security/certmanager 488.382s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestTLSOriginationSDS egress_gateways_tls_sds_test.go:37: Perform mTLS origination with an egress gateway egress_gateways_tls_sds_test.go:38: egress_gateways_tls_sds_test.go:38: STEP 1: Apply default SMCP and SMMR manifests egress_gateways_tls_sds_test.go:38: Wait for condition condition=Ready on smcp istio-system/basic... egress_gateways_tls_sds_test.go:38: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 egress_gateways_tls_sds_test.go:38: Wait for condition condition=Ready on smcp istio-system/basic... egress_gateways_tls_sds_test.go:38: FAILURE: Command failed: oc wait -n istio-system smcp/basic --for condition=Ready --timeout 10s error: timed out waiting for the condition on servicemeshcontrolplanes/basic error: exit status 1 egress_gateways_tls_sds_test.go:38: Wait for condition condition=Ready on smcp istio-system/basic... egress_gateways_tls_sds_test.go:38: SUCCESS: Condition condition=Ready met by smcp istio-system/basic egress_gateways_tls_sds_test.go:40: egress_gateways_tls_sds_test.go:40: STEP 2: Install sleep pod egress_gateways_tls_sds_test.go:42: Install app "sleep" in namespace "bookinfo" egress_gateways_tls_sds_test.go:42: Wait for app bookinfo/sleep to be ready egress_gateways_tls_sds_test.go:44: egress_gateways_tls_sds_test.go:44: STEP 3: Deploy nginx mTLS server and create secrets in the mesh namespace egress_gateways_tls_sds_test.go:45: Install app "nginx" in namespace "mesh-external" egress_gateways_tls_sds_test.go:45: Wait for app mesh-external/nginx to be ready egress_gateways_tls_sds_test.go:53: Send HTTP request to external nginx to verify mTLS origination egress_common.go:24: SUCCESS: Got expected 200 OK from http://my-nginx.mesh-external.svc.cluster.local test.go:77: test.go:82: Test completed in 30.37s (excluding cleanup) egress_gateways_tls_sds_test.go:29: egress_gateways_tls_sds_test.go:29: Performing cleanup egress_gateways_tls_sds_test.go:30: Deleting namespaces: [bookinfo] egress_gateways_tls_sds_test.go:30: Creating namespaces: [bookinfo] egress_gateways_tls_sds_test.go:31: Deleting namespaces: [mesh-external] egress_gateways_tls_sds_test.go:31: Creating namespaces: [mesh-external] egress_gateways_tls_sds_test.go:29: Cleanup completed in 24.67s --- PASS: TestTLSOriginationSDS (55.04s) PASS ok github.com/maistra/maistra-test-tool/pkg/tests/tasks/traffic/egress 55.464s DONE 2 runs, 110 tests, 20 skipped, 3 failures in 2609.519s test_helper_setup.go:64: Creating namespaces: [istio-system bookinfo foo bar legacy mesh-external] === RUN TestPluginCaCert install.go:32: cert-manager-operator is not installed, starting installation install.go:59: install.go:59: STEP 1: Create namespace for cert-manager-operator install.go:60: Creating namespaces: [cert-manager-operator] install.go:62: install.go:62: STEP 2: Install cert-manager-operator install.go:67: Waiting for cert-manager-operator to succeed install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 1/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 2/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 3/60 failed. Retrying... install.go:69: --- Attempt 4/60 successful; total time: 3.53s install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: SUCCESS: Condition jsonpath='{.status.phase}'=Succeeded met by csv cert-manager-operator/cert-manager-operator.v1.11.1 install.go:76: Pod cert-manager-operator-controller-manager-c5c6bddcf-5fqs8 in namespace cert-manager-operator is ready! install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 1/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 2/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 3/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 4/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 5/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 6/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 7/60 failed. Retrying in 4s... install.go:77: Pod cert-manager-5c5bc4f8f4-jlm8q in namespace cert-manager is ready! install.go:77: --- Attempt 8/60 successful; total time: 29.54s install.go:41: install.go:41: STEP 3: Create root ca plugin_ca_test.go:58: plugin_ca_test.go:58: STEP 4: Uninstall existing SMCP plugin_ca_test.go:59: Deleting namespaces: [istio-system] plugin_ca_test.go:59: Creating namespaces: [istio-system] plugin_ca_test.go:61: plugin_ca_test.go:61: STEP 5: Create intermediate CA certificate for Istio plugin_ca_test.go:64: plugin_ca_test.go:64: STEP 6: Deploy SMCP v2.4 and SMMR plugin_ca_test.go:66: Wait for condition condition=Ready on smcp istio-system/basic... plugin_ca_test.go:66: SUCCESS: Condition condition=Ready met by smcp istio-system/basic plugin_ca_test.go:68: plugin_ca_test.go:68: STEP 7: Verify that cacerts secret was detected plugin_ca_test.go:70: SUCCESS: Istiod detected cacerts secret correctly plugin_ca_test.go:76: plugin_ca_test.go:76: STEP 8: Deploy httpbin and sleep plugin_ca_test.go:77: Install app "httpbin" in namespace "foo" plugin_ca_test.go:77: Install app "sleep" in namespace "foo" plugin_ca_test.go:77: Wait for app foo/httpbin to be ready plugin_ca_test.go:77: Wait for app foo/sleep to be ready plugin_ca_test.go:79: plugin_ca_test.go:79: STEP 9: Check if httpbin returns 200 OK plugin_ca_test.go:81: SUCCESS: Got expected 200 OK from httpbin plugin_ca_test.go:91: plugin_ca_test.go:91: STEP 10: Check mTLS traffic from ingress gateway to httpbin plugin_ca_test.go:95: SUCCESS: received expected status code 200 plugin_ca_test.go:98: plugin_ca_test.go:98: STEP 11: Check current istiod generation plugin_ca_test.go:101: plugin_ca_test.go:101: STEP 12: Trigger CA cert rotation plugin_ca_test.go:104: plugin_ca_test.go:104: STEP 13: Wait until certificates reloaded plugin_ca_test.go:106: SUCCESS: Istiod detected cacerts secret correctly plugin_ca_test.go:119: plugin_ca_test.go:119: STEP 14: Trigger CA cert rotation plugin_ca_test.go:122: plugin_ca_test.go:122: STEP 15: Wait until certificates reloaded plugin_ca_test.go:124: SUCCESS: Istiod detected cacerts secret correctly plugin_ca_test.go:132: plugin_ca_test.go:132: STEP 16: Make sure that istiod was not restarted test.go:77: test.go:82: Test completed in 79.52s (excluding cleanup) plugin_ca_test.go:48: plugin_ca_test.go:48: Performing cleanup plugin_ca_test.go:52: Deleting namespaces: [foo] plugin_ca_test.go:52: Creating namespaces: [foo] install.go:49: Deleting namespaces: [cert-manager-operator] install.go:50: Deleting namespaces: [cert-manager] plugin_ca_test.go:48: Cleanup completed in 49.21s --- PASS: TestPluginCaCert (128.73s) PASS ok github.com/maistra/maistra-test-tool/pkg/tests/tasks/security/certmanager 129.183s === Skipped === SKIP: pkg/tests/non-dependant TestSMCPMultiple (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm TestIstiodPodFailsAfterRestarts (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm TestDeployOnInfraNodes (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm TestDiscoverySelectors (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm TestInitContainerNotRemovedDuringInjection (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm TestIOR (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm TestOperatorCanUpdatePrometheusConfigMap (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm TestRateLimiting (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm TestSMCPAddons (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm TestSMCPAnnotations (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm TestMustGather (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm TestSMMRAutoCreationAndDeletion (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm-federation TestMultiClusterFederationFailover (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm-federation TestFederation (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm-federation TestFederationDifferentCerts (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm/operator TestClusterWideMode (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm/operator TestOperatorCanReconcileSMCPWhenIstiodOffline (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/ossm/operator TestOperatorPodHonorsReadinessProbe (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/tasks/observability TestCustomPrometheus (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === SKIP: pkg/tests/tasks/observability TestOpenShiftMonitoring (0.00s) test.go:112: This test is being skipped because it is not part of the "interop" test group === Failed === FAIL: pkg/tests/tasks/security/certmanager TestPluginCaCert (145.94s) install.go:32: cert-manager-operator is not installed, starting installation install.go:59: install.go:59: STEP 1: Create namespace for cert-manager-operator install.go:60: Creating namespaces: [cert-manager-operator] install.go:62: install.go:62: STEP 2: Install cert-manager-operator install.go:67: Waiting for cert-manager-operator to succeed install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 1/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 2/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 3/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 4/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 5/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 6/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 7/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 8/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 9/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 10/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 11/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 12/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 13/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 14/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 15/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 16/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 17/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 18/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 19/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 20/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 21/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 22/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 23/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 24/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 25/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 26/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 27/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 28/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 29/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 30/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 31/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 32/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 33/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 34/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 35/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 36/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 37/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 38/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 39/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 40/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 41/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 42/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 43/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 44/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 45/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 46/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 47/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 48/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 49/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 50/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 51/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 52/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 53/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 54/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 55/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 56/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 57/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 58/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 59/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: Last attempt (60/60) failed. install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: FAILURE: Command failed: oc wait -n cert-manager-operator csv/cert-manager-operator.v1.11.1 --for jsonpath='{.status.phase}'=Succeeded --timeout 10s Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 install.go:75: FATAL: Command failed: oc describe -n cert-manager-operator csv/cert-manager-operator.v1.11.1 Error from server (NotFound): clusterserviceversions.operators.coreos.com "cert-manager-operator.v1.11.1" not found error: exit status 1 test.go:77: test.go:79: Test failed in 74.11s (excluding cleanup) test.go:95: Capturing cluster state using must-gather registry.redhat.io/openshift-service-mesh/istio-must-gather-rhel8:2.4 test.go:99: /go/src/maistra-test-tool/tests/result-20230802171548/v2.4/failures-must-gather/20230802173544-TestPluginCaCert plugin_ca_test.go:48: plugin_ca_test.go:48: Performing cleanup plugin_ca_test.go:52: Deleting namespaces: [foo] plugin_ca_test.go:52: Creating namespaces: [foo] install.go:49: Deleting namespaces: [cert-manager-operator] install.go:50: Deleting namespaces: [cert-manager] plugin_ca_test.go:48: Cleanup completed in 31.03s === FAIL: pkg/tests/tasks/security/certmanager TestPluginCaCert (re-run 1) (487.87s) install.go:32: cert-manager-operator is not installed, starting installation install.go:59: install.go:59: STEP 1: Create namespace for cert-manager-operator install.go:60: Creating namespaces: [cert-manager-operator] install.go:62: install.go:62: STEP 2: Install cert-manager-operator install.go:67: Waiting for cert-manager-operator to succeed install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 1/60 failed. Retrying... install.go:71: FAILURE: cert-manager-operator is not yet installed install.go:69: --- Attempt 2/60 failed. Retrying... install.go:69: --- Attempt 3/60 successful; total time: 2.44s install.go:75: Wait for condition jsonpath='{.status.phase}'=Succeeded on csv cert-manager-operator/cert-manager-operator.v1.11.1... install.go:75: SUCCESS: Condition jsonpath='{.status.phase}'=Succeeded met by csv cert-manager-operator/cert-manager-operator.v1.11.1 install.go:76: Pod cert-manager-operator-controller-manager-c5c6bddcf-z6tzf in namespace cert-manager-operator is ready! install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 1/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 2/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 3/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 4/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 5/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 6/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 7/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 8/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 9/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 10/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 11/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 12/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 13/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 14/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 15/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 16/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 17/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 18/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 19/60 failed. Retrying in 4s... install.go:77: FAILURE: no pods found using selector app=cert-manager in namespace cert-manager install.go:77: --- Attempt 20/60 failed. Retrying in 4s... install.go:77: FAILURE: more than one pod found using selector "app=cert-manager" in namespace cert-manager: [{cert-manager cert-manager-5c5bc4f8f4-kkvrj} {cert-manager cert-manager-84b449cdff-r4sf8}] install.go:77: --- Attempt 21/60 failed. Retrying in 4s... install.go:77: Pod cert-manager-5c5bc4f8f4-kkvrj in namespace cert-manager is ready! install.go:77: --- Attempt 22/60 successful; total time: 87.12s install.go:41: install.go:41: STEP 3: Create root ca plugin_ca_test.go:58: plugin_ca_test.go:58: STEP 4: Uninstall existing SMCP plugin_ca_test.go:59: Deleting namespaces: [istio-system] plugin_ca_test.go:59: Creating namespaces: [istio-system] plugin_ca_test.go:61: plugin_ca_test.go:61: STEP 5: Create intermediate CA certificate for Istio plugin_ca_test.go:64: plugin_ca_test.go:64: STEP 6: Deploy SMCP v2.4 and SMMR plugin_ca_test.go:66: Wait for condition condition=Ready on smcp istio-system/basic... plugin_ca_test.go:66: SUCCESS: Condition condition=Ready met by smcp istio-system/basic plugin_ca_test.go:68: plugin_ca_test.go:68: STEP 7: Verify that cacerts secret was detected plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 1/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 2/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 3/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 4/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 5/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 6/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 7/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 8/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 9/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 10/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 11/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 12/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 13/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 14/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 15/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 16/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 17/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 18/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 19/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 20/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 21/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 22/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 23/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 24/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 25/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 26/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 27/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 28/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 29/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 30/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 31/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 32/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 33/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 34/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 35/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 36/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 37/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 38/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 39/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 40/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 41/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 42/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 43/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 44/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 45/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 46/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 47/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 48/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 49/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 50/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 51/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 52/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 53/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 54/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 55/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 56/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 57/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 58/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found plugin_ca_test.go:69: --- Attempt 59/60 failed. Retrying... plugin_ca_test.go:70: FAILURE: Istiod did not detect cacerts secret; expected to find the string 'Use plugged-in cert at etc/cacerts/tls.key' in the output, but it wasn't found; full output: 2023-08-02T17:52:08.955480Z info FLAG: --caCertFile="" 2023-08-02T17:52:08.955504Z info FLAG: --clusterAliases="[]" 2023-08-02T17:52:08.955508Z info FLAG: --clusterID="Kubernetes" 2023-08-02T17:52:08.955510Z info FLAG: --clusterRegistriesNamespace="istio-system" 2023-08-02T17:52:08.955513Z info FLAG: --configDir="" 2023-08-02T17:52:08.955515Z info FLAG: --ctrlz_address="localhost" 2023-08-02T17:52:08.955518Z info FLAG: --ctrlz_port="9876" 2023-08-02T17:52:08.955521Z info FLAG: --domain="cluster.local" 2023-08-02T17:52:08.955525Z info FLAG: --enableCRDScan="false" 2023-08-02T17:52:08.955527Z info FLAG: --enableIngressClassName="false" 2023-08-02T17:52:08.955530Z info FLAG: --enableNodeAccess="false" 2023-08-02T17:52:08.955533Z info FLAG: --federationAddr=":8188" 2023-08-02T17:52:08.955535Z info FLAG: --grpcAddr=":15010" 2023-08-02T17:52:08.955538Z info FLAG: --help="false" 2023-08-02T17:52:08.955540Z info FLAG: --httpAddr=":8080" 2023-08-02T17:52:08.955543Z info FLAG: --httpsAddr=":15017" 2023-08-02T17:52:08.955547Z info FLAG: --keepaliveInterval="30s" 2023-08-02T17:52:08.955550Z info FLAG: --keepaliveMaxServerConnectionAge="30m0s" 2023-08-02T17:52:08.955553Z info FLAG: --keepaliveTimeout="10s" 2023-08-02T17:52:08.955555Z info FLAG: --kubeconfig="" 2023-08-02T17:52:08.955558Z info FLAG: --kubernetesApiBurst="160" 2023-08-02T17:52:08.955562Z info FLAG: --kubernetesApiQPS="80" 2023-08-02T17:52:08.955565Z info FLAG: --logKubernetesApiRequests="false" 2023-08-02T17:52:08.955568Z info FLAG: --log_as_json="false" 2023-08-02T17:52:08.955570Z info FLAG: --log_caller="" 2023-08-02T17:52:08.955573Z info FLAG: --log_output_level="default:info" 2023-08-02T17:52:08.955576Z info FLAG: --log_rotate="" 2023-08-02T17:52:08.955579Z info FLAG: --log_rotate_max_age="30" 2023-08-02T17:52:08.955581Z info FLAG: --log_rotate_max_backups="1000" 2023-08-02T17:52:08.955584Z info FLAG: --log_rotate_max_size="104857600" 2023-08-02T17:52:08.955587Z info FLAG: --log_stacktrace_level="default:none" 2023-08-02T17:52:08.955594Z info FLAG: --log_target="[stdout]" 2023-08-02T17:52:08.955596Z info FLAG: --memberRollName="default" 2023-08-02T17:52:08.955599Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2023-08-02T17:52:08.955602Z info FLAG: --monitoringAddr=":15014" 2023-08-02T17:52:08.955604Z info FLAG: --namespace="istio-system" 2023-08-02T17:52:08.955607Z info FLAG: --networksConfig="./etc/istio/config/meshNetworks" 2023-08-02T17:52:08.955609Z info FLAG: --profile="true" 2023-08-02T17:52:08.955614Z info FLAG: --registries="[Kubernetes]" 2023-08-02T17:52:08.955617Z info FLAG: --resync="1m0s" 2023-08-02T17:52:08.955619Z info FLAG: --secureGRPCAddr=":15012" 2023-08-02T17:52:08.955622Z info FLAG: --shutdownDuration="10s" 2023-08-02T17:52:08.955625Z info FLAG: --tls-cipher-suites="[]" 2023-08-02T17:52:08.955628Z info FLAG: --tlsCertFile="" 2023-08-02T17:52:08.955630Z info FLAG: --tlsKeyFile="" 2023-08-02T17:52:08.955633Z info FLAG: --vklog="0" 2023-08-02T17:52:08.964726Z info initializing Istiod admin server 2023-08-02T17:52:08.965537Z info klog Config not found: /var/run/secrets/remote/config 2023-08-02T17:52:08.966448Z info starting HTTP service at [::]:8080 2023-08-02T17:52:08.966954Z info smmr Member namespace list updated: [] 2023-08-02T17:52:08.977520Z info initializing mesh configuration ./etc/istio/config/mesh 2023-08-02T17:52:08.984430Z info controllers starting controller=configmap istio-basic 2023-08-02T17:52:08.984642Z info Loaded MeshNetworks config from Kubernetes API server. 2023-08-02T17:52:08.984658Z info mesh networks configuration updated to: { } 2023-08-02T17:52:08.985812Z info Loaded MeshConfig config from Kubernetes API server. 2023-08-02T17:52:08.986210Z info mesh configuration updated to: { "proxyListenPort": 15001, "connectTimeout": "10s", "protocolDetectionTimeout": "0s", "ingressClass": "istio", "ingressService": "istio-ingressgateway", "ingressControllerMode": "OFF", "enableTracing": true, "defaultConfig": { "configPath": "./etc/istio/proxy", "binaryPath": "/usr/local/bin/envoy", "serviceCluster": "istio-proxy", "drainDuration": "45s", "parentShutdownDuration": "60s", "discoveryAddress": "istiod-basic.istio-system.svc:15012", "proxyAdminPort": 15000, "controlPlaneAuthPolicy": "MUTUAL_TLS", "statNameLength": 189, "concurrency": 2, "tracing": { "zipkin": { "address": "zipkin.istio-system:9411" } }, "proxyMetadata": { "ISTIO_META_DNS_AUTO_ALLOCATE": "true", "ISTIO_META_DNS_CAPTURE": "true", "PROXY_XDS_VIA_AGENT": "true" }, "statusPort": 15020, "terminationDrainDuration": "5s" }, "outboundTrafficPolicy": { "mode": "ALLOW_ANY" }, "enableAutoMtls": true, "trustDomain": "cluster.local", "defaultServiceExportTo": [ "*" ], "defaultVirtualServiceExportTo": [ "*" ], "defaultDestinationRuleExportTo": [ "*" ], "rootNamespace": "istio-system", "localityLbSetting": { "enabled": true }, "dnsRefreshRate": "300s", "thriftConfig": { }, "enablePrometheusMerge": true, "extensionProviders": [ { "name": "prometheus", "prometheus": { } }, { "name": "stackdriver", "stackdriver": { } }, { "name": "envoy", "envoyFileAccessLog": { "path": "/dev/stdout" } } ], "defaultProviders": { } } 2023-08-02T17:52:08.992546Z info initializing mesh networks from mesh config watcher 2023-08-02T17:52:08.992641Z info mesh configuration: { "proxyListenPort": 15001, "connectTimeout": "10s", "protocolDetectionTimeout": "0s", "ingressClass": "istio", "ingressService": "istio-ingressgateway", "ingressControllerMode": "OFF", "enableTracing": true, "defaultConfig": { "configPath": "./etc/istio/proxy", "binaryPath": "/usr/local/bin/envoy", "serviceCluster": "istio-proxy", "drainDuration": "45s", "parentShutdownDuration": "60s", "discoveryAddress": "istiod-basic.istio-system.svc:15012", "proxyAdminPort": 15000, "controlPlaneAuthPolicy": "MUTUAL_TLS", "statNameLength": 189, "concurrency": 2, "tracing": { "zipkin": { "address": "zipkin.istio-system:9411" } }, "proxyMetadata": { "ISTIO_META_DNS_AUTO_ALLOCATE": "true", "ISTIO_META_DNS_CAPTURE": "true", "PROXY_XDS_VIA_AGENT": "true" }, "statusPort": 15020, "terminationDrainDuration": "5s" }, "outboundTrafficPolicy": { "mode": "ALLOW_ANY" }, "enableAutoMtls": true, "trustDomain": "cluster.local", "defaultServiceExportTo": [ "*" ], "defaultVirtualServiceExportTo": [ "*" ], "defaultDestinationRuleExportTo": [ "*" ], "rootNamespace": "istio-system", "localityLbSetting": { "enabled": true }, "dnsRefreshRate": "300s", "thriftConfig": { }, "enablePrometheusMerge": true, "extensionProviders": [ { "name": "prometheus", "prometheus": { } }, { "name": "stackdriver", "stackdriver": { } }, { "name": "envoy", "envoyFileAccessLog": { "path": "/dev/stdout" } } ], "defaultProviders": { } } 2023-08-02T17:52:08.992651Z info version: OSSM_2.4.1-1-466b6910fb9c61ad1d30759b511e1c31b7cf7dd3-Clean 2023-08-02T17:52:08.992830Z info flags: { "ServerOptions": { "HTTPAddr": ":8080", "HTTPSAddr": ":15017", "GRPCAddr": ":15010", "MonitoringAddr": ":15014", "EnableProfiling": true, "TLSOptions": { "CaCertFile": "", "CertFile": "", "KeyFile": "", "TLSCipherSuites": null, "CipherSuits": null }, "SecureGRPCAddr": ":15012", "FederationAddr": ":8188" }, "InjectionOptions": { "InjectionDirectory": "./var/lib/istio/inject" }, "PodName": "istiod-basic-6dbb5fcd4d-g82lq", "Namespace": "istio-system", "Revision": "basic", "MeshConfigFile": "./etc/istio/config/mesh", "Network": "", "NetworksConfigFile": "./etc/istio/config/meshNetworks", "RegistryOptions": { "FileDir": "", "Registries": [ "Kubernetes" ], "KubeOptions": { "SystemNamespace": "", "MeshServiceController": null, "ResyncPeriod": 60000000000, "DomainSuffix": "cluster.local", "MemberRollName": "default", "ClusterID": "Kubernetes", "ClusterAliases": {}, "Metrics": null, "XDSUpdater": null, "NetworksWatcher": null, "MeshWatcher": null, "EndpointMode": 1, "KubernetesAPIQPS": 80, "KubernetesAPIBurst": 160, "LogKubernetesAPIRequests": false, "SyncTimeout": 0, "DiscoveryNamespacesFilter": null, "EnableCRDScan": false, "EnableNodeAccess": false, "EnableIngressClassName": false }, "ClusterRegistriesNamespace": "istio-system", "KubeConfig": "", "DistributionCacheRetention": 60000000000, "DistributionTrackingEnabled": false }, "CtrlZOptions": { "Port": 9876, "Address": "localhost" }, "KeepaliveOptions": { "Time": 30000000000, "Timeout": 10000000000, "MaxServerConnectionAge": 1800000000000, "MaxServerConnectionAgeGrace": 10000000000 }, "ShutdownDuration": 10000000000, "JwtRule": "" } 2023-08-02T17:52:08.992841Z info initializing mesh handlers 2023-08-02T17:52:08.992873Z info model reloading network gateways 2023-08-02T17:52:08.992906Z info creating CA and initializing public key 2023-08-02T17:52:08.992944Z info Using istiod file format for signing ca files 2023-08-02T17:52:08.992959Z info Use self-signed certificate as the CA certificate 2023-08-02T17:52:08.996460Z info pkica Failed to get secret (error: secrets "istio-ca-secret" not found), will create one 2023-08-02T17:52:09.244488Z info pkica Using self-generated public key: -----BEGIN CERTIFICATE----- MIIC/TCCAeWgAwIBAgIRAJW/d5Dfxyi16KdUj1aCa2owDQYJKoZIhvcNAQELBQAw GDEWMBQGA1UEChMNY2x1c3Rlci5sb2NhbDAeFw0yMzA4MDIxNzUyMDlaFw0zMzA3 MzAxNzUyMDlaMBgxFjAUBgNVBAoTDWNsdXN0ZXIubG9jYWwwggEiMA0GCSqGSIb3 DQEBAQUAA4IBDwAwggEKAoIBAQCwo3cT0ohZYRUl/yV76gBNnUzsbA+DhUXIBcEN OiHIYpX3wfc/beVLaGS+0hqBdINXWO/O6qsN54T+96jfdJ/QnvidUVNdiqIQYXlQ srHFMH1W28o7mn8nD2t62sr5+CSCjTYROnFGDjslEZ/IOBkTfwkHKHoGHCwnUKuk rAaKAJWtbPBX152EmeeNi0sQ+bEPo/CoNjJnFeT0oAq9FvnVUGE/E1ku2gl9Vw4s JlxMdmXn7Sj+tj2gstPUJxs9doNamTJP8VhhpQIeZjiIC2U0vZpP41+sRixyWq0I FzpNkC/kkOiTrK+xJjJng+mv+RjRRfhixsEzYr53RrMFnvibAgMBAAGjQjBAMA4G A1UdDwEB/wQEAwICBDAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBT2iekZQWiQ n3/Jj1KdLjJ6e1zXbzANBgkqhkiG9w0BAQsFAAOCAQEAOxAlVjwfH38cZShJzPfO HRZIfFCt3Ac02QIqV+Kbmt7LeKs2/EKWJ57iovNJjuHHBdm1Eyrlp6prkWju6mnN oTQ0ANjoELwKK933gQbXQQ0+btCyiPsMHWVa5UMFZXMMLGi8bHa2Qs2acmc6EENY ZaeoJzQXW3ZwbIWDrkZzSKOjddSyGpJUQgrLw7HxJlykul2rxre7XMqNWt7rLOAy KiXAKSSCoqzM9pQzMe2vimhIrMnEQFpElAlr8SkrGzZKiV1aAibclPT+4BPo341o ZMl29eCuYTjMhmyu7WyREJ6n6/WRohxCKs/H4glMPBmf4/VFwBZ+adr5lZXqMNAG nw== -----END CERTIFICATE----- 2023-08-02T17:52:09.244522Z info rootcertrotator Set up back off time 59m39s to start rotator. 2023-08-02T17:52:09.244596Z info initializing Istiod DNS certificates host: istiod-basic.istio-system.svc, custom host: 2023-08-02T17:52:09.244838Z info rootcertrotator Jitter is enabled, wait 59m39s before starting root cert rotator. 2023-08-02T17:52:09.307372Z info Generating istiod-signed cert for [istiod-basic.istio-system.svc istiod.istio-system.svc istiod-remote.istio-system.svc istio-pilot.istio-system.svc]: -----BEGIN CERTIFICATE----- MIIDiTCCAnGgAwIBAgIQG3RH7C5iicMZ2WGHiEtdXDANBgkqhkiG9w0BAQsFADAY MRYwFAYDVQQKEw1jbHVzdGVyLmxvY2FsMB4XDTIzMDgwMjE3NTAwOVoXDTMzMDcz MDE3NTIwOVowADCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKDWJm06 W5McTw/kHLTK2MljNzVXNAbQTgvqEMQ8VyUaIldqRCm0OeR6R782esFhziQiLAnl S+HpKVqtQo4LsqP3vHcuCwrrihqQSR6GPceE1TkFwbj6BIFfsDyrL1u9x/ZVpPFS 3P8M5kq/sAVNp+sm/Z35FI1SRuh/tGIVYXEWjaf1hAQsu82goe6JrqNtSED1Y5yZ d9Qs2NnBA3H9XyiTlmiCOac3g8sCGnX6DsneoK/YBO7tOmIU4tRoum3K8IQTs+fa bgvgNVwsBayoKxmo/BavAqiuLd02HcnbuQNMjED7ZHv/ifRskVq1w5qP3x+HDh3Y DFR0yXqNOppbqRMCAwEAAaOB5jCB4zAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYw FAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAU 9onpGUFokJ9/yY9SnS4yentc128wgYIGA1UdEQEB/wR4MHaCHWlzdGlvZC1iYXNp Yy5pc3Rpby1zeXN0ZW0uc3Zjghdpc3Rpb2QuaXN0aW8tc3lzdGVtLnN2Y4IeaXN0 aW9kLXJlbW90ZS5pc3Rpby1zeXN0ZW0uc3Zjghxpc3Rpby1waWxvdC5pc3Rpby1z eXN0ZW0uc3ZjMA0GCSqGSIb3DQEBCwUAA4IBAQA52eLnViZSkRrCKQiqKZU/Uw+a JDTBFEugZkaSxi4j0GB+8L9jOpDtw8fB7VES8vihlmWbjibqJn3m8TjvCB222fN2 Z3Lare1graUMcuLr5Bg33lMG2IgkAcCpcrSDuxEAvuGA9VCx9CErawUHyl16uuKB zU93MmBExOzEZMybHh+CJRq5Bgl71vKfN4u96V8miDPRCIbf8/l/oCIDL+7pcCT+ baEd0bWrhBnigkg7csjV1gP5HQOveJdIH3owOv5z+NrAkTassrZYNvhbANWDniC/ hkcm5eP+I3bvcvFhX+TpKcTBEq1tkR5hFx9ZAq7RaXWIGIdz7DK32GD2QBxL -----END CERTIFICATE----- 2023-08-02T17:52:09.307412Z info Using istiod file format for signing ca files 2023-08-02T17:52:09.307422Z info No plugged-in cert at etc/cacerts/ca-key.pem; self-signed cert is used 2023-08-02T17:52:09.307529Z info x509 cert - Issuer: "O=cluster.local", Subject: "", SN: 1b7447ec2e6289c319d96187884b5d5c, NotBefore: "2023-08-02T17:50:09Z", NotAfter: "2033-07-30T17:52:09Z" 2023-08-02T17:52:09.307539Z info Istiod certificates are reloaded 2023-08-02T17:52:09.307545Z info initializing controllers 2023-08-02T17:52:09.307597Z info spiffe Added 1 certs to trust domain cluster.local in peer cert verifier 2023-08-02T17:52:09.307608Z info Go Cipher suites are [] 2023-08-02T17:52:09.307612Z info OpenSSL Cipher suites are [] 2023-08-02T17:52:09.307616Z info Go ECDH Curves are [] 2023-08-02T17:52:09.307619Z info OpenSSL ECDH Curves are [] 2023-08-02T17:52:09.307916Z warn federation ResyncPeriod not specified, defaulting to 1m0s component=federation-discovery-controller 2023-08-02T17:52:09.307963Z info No certificates specified, skipping K8S DNS certificate controller 2023-08-02T17:52:09.308232Z info Adding Kubernetes registry adapter 2023-08-02T17:52:09.308295Z info spiffe Added 1 certs to trust domain cluster.local in peer cert verifier 2023-08-02T17:52:09.308303Z info initializing secure discovery service 2023-08-02T17:52:09.308334Z info initializing secure webhook server for istiod webhooks 2023-08-02T17:52:09.316576Z info initializing sidecar injector 2023-08-02T17:52:09.329287Z info initializing config validator 2023-08-02T17:52:09.329301Z info initializing registry event handlers 2023-08-02T17:52:09.329319Z info starting discovery service 2023-08-02T17:52:09.331065Z info Starting Istiod Server with primary cluster Kubernetes 2023-08-02T17:52:09.331141Z info ControlZ available at 127.0.0.1:9876 2023-08-02T17:52:09.331213Z info kube Initializing Kubernetes service registry "Kubernetes" 2023-08-02T17:52:09.331237Z info federation Starting controller component=federation-discovery-controller 2023-08-02T17:52:09.331279Z info kube should join leader-election for cluster Kubernetes: false 2023-08-02T17:52:09.331309Z info initializing Kubernetes credential reader for cluster Kubernetes 2023-08-02T17:52:09.331322Z info federation Starting controller component=federation-exports-controller 2023-08-02T17:52:09.331287Z info klog attempting to acquire leader lease istio-system/servicemesh-federation... 2023-08-02T17:52:09.331321Z info federation Starting controller component=federation-imports-controller 2023-08-02T17:52:09.332312Z info Starting ADS server 2023-08-02T17:52:09.332391Z info Starting IstioD CA 2023-08-02T17:52:09.332416Z info JWT policy is first-party-jwt 2023-08-02T17:52:09.332550Z info Istiod CA has started 2023-08-02T17:52:09.332827Z info controllers starting controller=healthcheck 2023-08-02T17:52:09.333289Z info klog Waiting for caches to sync for smmr 2023-08-02T17:52:09.333431Z info controllers starting controller=unregister_workloadentry 2023-08-02T17:52:09.333523Z info kube Starting Pilot K8S CRD controller controller=crd-controller 2023-08-02T17:52:09.333550Z info kube controller "networking.istio.io/v1alpha3/EnvoyFilter" is syncing... controller=crd-controller 2023-08-02T17:52:09.333712Z info Starting multicluster remote secrets controller 2023-08-02T17:52:09.335801Z info kube controller "networking.istio.io/v1alpha3/EnvoyFilter" is syncing... controller=crd-controller 2023-08-02T17:52:09.338855Z info controllers starting controller=configmap istio-sidecar-injector-basic 2023-08-02T17:52:09.340355Z info kube controller "networking.istio.io/v1alpha3/VirtualService" is syncing... controller=crd-controller 2023-08-02T17:52:09.340360Z info multicluster remote secrets controller cache synced in 6.680753ms 2023-08-02T17:52:09.340397Z info controllers starting controller=multicluster secret 2023-08-02T17:52:09.346539Z info klog successfully acquired lease istio-system/servicemesh-federation 2023-08-02T17:52:09.346584Z info leader election lock obtained: servicemesh-federation 2023-08-02T17:52:09.348619Z info kube controller "networking.istio.io/v1alpha3/DestinationRule" is syncing... controller=crd-controller 2023-08-02T17:52:09.365451Z info kube controller "networking.istio.io/v1beta1/ProxyConfig" is syncing... controller=crd-controller 2023-08-02T17:52:09.397950Z info kube controller "networking.istio.io/v1alpha3/EnvoyFilter" is syncing... controller=crd-controller 2023-08-02T17:52:09.434374Z info klog Caches are synced for smmr 2023-08-02T17:52:09.462810Z info kube controller "networking.istio.io/v1alpha3/EnvoyFilter" is syncing... controller=crd-controller 2023-08-02T17:52:09.467917Z info smmr Cache synced for listener "dynamic-informers" 2023-08-02T17:52:09.467926Z info smmr Cache synced for listener "namespace logger" 2023-08-02T17:52:09.467941Z info smmr Cache synced for listener "istio-informers" 2023-08-02T17:52:09.467968Z info smmr Cache synced for listener "openshift-route-informers" 2023-08-02T17:52:09.467990Z info smmr Cache synced for listener "kubernetes-informers" 2023-08-02T17:52:09.467977Z info smmr Member namespace list updated: [foo istio-system] 2023-08-02T17:52:09.531767Z info smmr Cache synced for listener "MaistraDiscoveryNamespacesFilter" 2023-08-02T17:52:09.531952Z info federation Controller synced in 200.626481ms component=federation-exports-controller 2023-08-02T17:52:09.531990Z info federation Starting workers component=federation-exports-controller 2023-08-02T17:52:09.531964Z info federation Controller synced in 200.735324ms component=federation-discovery-controller 2023-08-02T17:52:09.532003Z info federation Starting workers component=federation-discovery-controller 2023-08-02T17:52:09.533187Z info federation Controller synced in 201.868272ms component=federation-imports-controller 2023-08-02T17:52:09.533221Z info federation Starting workers component=federation-imports-controller 2023-08-02T17:52:09.533256Z info Waiting for caches to be synced 2023-08-02T17:52:09.533287Z info kube joining leader-election for istio-namespace-controller-election in istio-system on cluster Kubernetes 2023-08-02T17:52:09.533344Z info klog attempting to acquire leader lease istio-system/istio-namespace-controller-election... 2023-08-02T17:52:09.542629Z info klog successfully acquired lease istio-system/istio-namespace-controller-election 2023-08-02T17:52:09.542681Z info leader election lock obtained: istio-namespace-controller-election 2023-08-02T17:52:09.542726Z info kube starting namespace controller for cluster Kubernetes 2023-08-02T17:52:09.542854Z info controllers starting controller=namespace controller 2023-08-02T17:52:09.542867Z info smmr Cache synced for listener "namespace-controller" 2023-08-02T17:52:09.549612Z error controllers error handling foo/istio-ca-root-cert, and retry budget exceeded: error when updating configmap istio-ca-root-cert: Operation cannot be fulfilled on configmaps "istio-ca-root-cert": the object has been modified; please apply your changes to the latest version and try again controller=namespace controller 2023-08-02T17:52:09.561996Z info ads Incremental push, service istiod-basic.istio-system.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2023-08-02T17:52:09.562016Z info kube kube controller for Kubernetes synced after 228.685698ms 2023-08-02T17:52:09.563090Z info kube Pilot K8S CRD controller synced in 229.624862ms controller=crd-controller 2023-08-02T17:52:09.565133Z info All controller caches have been synced up in 31.875076ms 2023-08-02T17:52:09.663322Z info ads Push debounce stable[1] 18 for config Secret/foo/builder-dockercfg-mjwts and 16 more configs: 100.220238ms since last change, 179.370149ms since last push, full=true 2023-08-02T17:52:09.663638Z info ads XDS: Pushing:2023-08-02T17:52:09Z/1 Services:1 ConnectedEndpoints:0 Version:2023-08-02T17:52:09Z/1 2023-08-02T17:52:09.692473Z info ads All caches have been synced up in 738.873102ms, marking server ready 2023-08-02T17:52:09.692568Z info starting secure gRPC discovery service at [::]:15012 2023-08-02T17:52:09.692621Z info starting gRPC discovery service at [::]:15010 2023-08-02T17:52:09.692653Z info starting webhook service at [::]:15017 2023-08-02T17:52:09.692628Z info federation starting federation service discovery at [::]:8188 component=federation-server 2023-08-02T17:52:10.035907Z info ads Full push, new service istio-system/istiod-basic.istio-system.svc.cluster.local 2023-08-02T17:52:10.136234Z info ads Push debounce stable[2] 1 for config ServiceEntry/istio-system/istiod-basic.istio-system.svc.cluster.local: 100.270839ms since last change, 100.270548ms since last push, full=true 2023-08-02T17:52:10.136590Z info ads XDS: Pushing:2023-08-02T17:52:10Z/2 Services:1 ConnectedEndpoints:0 Version:2023-08-02T17:52:10Z/2 2023-08-02T17:52:10.498400Z info ads Incremental push, service istio-ingressgateway.istio-system.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2023-08-02T17:52:10.537216Z info Sidecar injection request for istio-system/istio-ingressgateway-86497c6c97-***** (actual name not yet known) 2023-08-02T17:52:10.537239Z info Skipping istio-system/istio-ingressgateway-86497c6c97-***** (actual name not yet known) due to policy check 2023-08-02T17:52:10.598731Z info ads Push debounce stable[3] 10 for config DestinationRule/istio-system/default and 7 more configs: 100.223118ms since last change, 318.200148ms since last push, full=true 2023-08-02T17:52:10.600333Z info ads XDS: Pushing:2023-08-02T17:52:10Z/3 Services:2 ConnectedEndpoints:0 Version:2023-08-02T17:52:10Z/3 2023-08-02T17:52:11.274929Z warn Insecure first-party-jwt option used to validate token; use third-party-jwt 2023-08-02T17:52:11.281125Z info ads ADS: new connection for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system-1 2023-08-02T17:52:11.281936Z info ads CDS: PUSH request for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:9 size:8.4kB cached:0/8 2023-08-02T17:52:11.290123Z info ads EDS: PUSH request for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:8 size:1.0kB empty:3 cached:0/8 2023-08-02T17:52:11.417699Z warn Insecure first-party-jwt option used to validate token; use third-party-jwt 2023-08-02T17:52:11.428270Z info ads LDS: PUSH request for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:0 size:0B 2023-08-02T17:52:11.968653Z info http: TLS handshake error from 10.128.0.31:40590: remote error: tls: bad certificate 2023-08-02T17:52:12.071511Z info ads Incremental push, service istio-ingressgateway.istio-system.svc.cluster.local at shard Kubernetes/Kubernetes has no endpoints 2023-08-02T17:52:12.084496Z info ads CDS: PUSH for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:9 size:8.4kB cached:0/8 2023-08-02T17:52:12.084673Z info ads EDS: PUSH for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:8 size:1.0kB empty:3 cached:0/8 2023-08-02T17:52:12.084750Z info ads LDS: PUSH for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:0 size:0B 2023-08-02T17:52:12.100068Z info ads Full push, new service istio-system/istio-ingressgateway.istio-system.svc.cluster.local 2023-08-02T17:52:12.200868Z info ads Push debounce stable[4] 2 for config ServiceEntry/istio-system/istio-ingressgateway.istio-system.svc.cluster.local: 100.71405ms since last change, 129.301171ms since last push, full=true 2023-08-02T17:52:12.201129Z info ads XDS: Pushing:2023-08-02T17:52:12Z/4 Services:2 ConnectedEndpoints:1 Version:2023-08-02T17:52:12Z/4 2023-08-02T17:52:12.201403Z info ads CDS: PUSH for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:9 size:8.6kB cached:5/8 2023-08-02T17:52:12.201523Z info ads EDS: PUSH INC for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:3 size:622B empty:0 cached:0/3 2023-08-02T17:52:12.201568Z info ads LDS: PUSH for node:istio-ingressgateway-86497c6c97-kh5nc.istio-system resources:0 size:0B 2023-08-02T17:52:15.750323Z info ads Push debounce stable[5] 1 for config Secret/istio-system/cacerts-2r8sg: 100.855825ms since last change, 100.855554ms since last push, full=false 2023-08-02T17:52:15.750376Z info ads XDS: Incremental Pushing:2023-08-02T17:52:12Z/4 ConnectedEndpoints:1 Version:2023-08-02T17:52:12Z/4 2023-08-02T17:52:20.570295Z info http: TLS handshake error from 10.128.0.31:53156: remote error: tls: bad certificate 2023-08-02T17:52:20.658682Z info ads Push debounce stable[6] 2 for config Secret/istio-system/cacerts and 1 more configs: 100.317589ms since last change, 115.676559ms since last push, full=false 2023-08-02T17:52:20.658731Z info ads XDS: Incremental Pushing:2023-08-02T17:52:12Z/4 ConnectedEndpoints:1 Version:2023-08-02T17:52:12Z/4 2023-08-02T17:52:21.576235Z info http: TLS handshake error from 10.128.0.31:53158: remote error: tls: bad certificate 2023-08-02T17:52:23.582662Z info http: TLS handshake error from 10.128.0.31:53172: remote error: tls: bad certificate 2023-08-02T17:52:23.650448Z info http: TLS handshake error from 10.128.0.31:53180: remote error: tls: bad certificate 2023-08-02T17:52:24.655854Z info http: TLS handshake error from 10.128.0.31:53194: remote error: tls: bad certificate 2023-08-02T17:52:26.662109Z info http: TLS handshake error from 10.128.0.31:53200: remote error: tls: bad certificate 2023-08-02T17:52:26.840067Z info http: TLS handshake error from 10.128.0.31:53202: remote error: tls: bad certificate 2023-08-02T17:52:27.845300Z info http: TLS handshake error from 10.128.0.31:47942: remote error: tls: bad certificate 2023-08-02T17:52:29.851044Z info http: TLS handshake error from 10.128.0.31:47946: remote error: tls: bad certificate 2023-08-02T17:52:29.869172Z info http: TLS handshake error from 10.128.0.31:47948: remote error: tls: bad certificate 2023-08-02T17:52:30.874619Z info http: TLS handshake error from 10.128.0.31:47964: remote error: tls: bad certificate 2023-08-02T17:52:32.880338Z info http: TLS handshake error from 10.128.0.31:47978: remote error: tls: bad certificate plugin_ca_test.go:69: Last attempt (60/60) failed. plugin_ca_test.go:76: plugin_ca_test.go:76: STEP 8: Deploy httpbin and sleep plugin_ca_test.go:77: Install app "httpbin" in namespace "foo" plugin_ca_test.go:77: Install app "sleep" in namespace "foo" plugin_ca_test.go:77: Wait for app foo/httpbin to be ready plugin_ca_test.go:77: FATAL: Command failed: kubectl -n foo rollout status deploy/httpbin --timeout=4m0s Waiting for deployment "httpbin" rollout to finish: 0 out of 1 new replicas have been updated... error: timed out waiting for the condition error: exit status 1 test.go:77: test.go:79: Test failed in 436.34s (excluding cleanup) test.go:95: Capturing cluster state using must-gather registry.redhat.io/openshift-service-mesh/istio-must-gather-rhel8:2.4 test.go:99: /go/src/maistra-test-tool/tests/result-20230802171548/v2.4/failures-must-gather/20230802175731-TestPluginCaCert plugin_ca_test.go:48: plugin_ca_test.go:48: Performing cleanup plugin_ca_test.go:52: Deleting namespaces: [foo] plugin_ca_test.go:52: Creating namespaces: [foo] install.go:49: Deleting namespaces: [cert-manager-operator] install.go:50: Deleting namespaces: [cert-manager] plugin_ca_test.go:48: Cleanup completed in 26.82s === FAIL: pkg/tests/tasks/traffic/egress TestTLSOriginationSDS (116.76s) egress_gateways_tls_sds_test.go:37: Perform mTLS origination with an egress gateway egress_gateways_tls_sds_test.go:38: egress_gateways_tls_sds_test.go:38: STEP 1: Apply default SMCP and SMMR manifests egress_gateways_tls_sds_test.go:38: Wait for condition condition=Ready on smcp istio-system/basic... egress_gateways_tls_sds_test.go:38: SUCCESS: Condition condition=Ready met by smcp istio-system/basic egress_gateways_tls_sds_test.go:40: egress_gateways_tls_sds_test.go:40: STEP 2: Install sleep pod egress_gateways_tls_sds_test.go:42: Install app "sleep" in namespace "bookinfo" egress_gateways_tls_sds_test.go:42: Wait for app bookinfo/sleep to be ready egress_gateways_tls_sds_test.go:44: egress_gateways_tls_sds_test.go:44: STEP 3: Deploy nginx mTLS server and create secrets in the mesh namespace egress_gateways_tls_sds_test.go:45: Install app "nginx" in namespace "mesh-external" egress_gateways_tls_sds_test.go:45: FATAL: Command failed: kubectl -n mesh-external delete secret nginx-ca-certs --ignore-not-found The connection to the server api.ci-op-11b996y7-8c5aa.cspilp.interop.ccitredhat.com:6443 was refused - did you specify the right host or port? error: exit status 1 test.go:77: test.go:79: Test failed in 12.72s (excluding cleanup) test.go:95: Capturing cluster state using must-gather registry.redhat.io/openshift-service-mesh/istio-must-gather-rhel8:2.4 test.go:99: /go/src/maistra-test-tool/tests/result-20230802171548/v2.4/failures-must-gather/20230802174300-TestTLSOriginationSDS egress_gateways_tls_sds_test.go:29: egress_gateways_tls_sds_test.go:29: Performing cleanup egress_gateways_tls_sds_test.go:30: Deleting namespaces: [bookinfo] egress_gateways_tls_sds_test.go:30: Creating namespaces: [bookinfo] egress_gateways_tls_sds_test.go:31: Deleting namespaces: [mesh-external] egress_gateways_tls_sds_test.go:31: Creating namespaces: [mesh-external] egress_gateways_tls_sds_test.go:29: Cleanup completed in 41.21s DONE 3 runs, 111 tests, 20 skipped, 3 failures in 2739.252s ====== JUnit report file(s) v2.4: /go/src/maistra-test-tool/tests/result-20230802171548/v2.4/report.xml ====== Test summary v2.4: DONE 3 runs, 111 tests, 20 skipped, 3 failures in 2739.252s real 45m40.985s user 4m50.297s sys 1m22.158s Copying logs and xmls to /logs/artifacts kubectl delete ns istio-system bookinfo foo bar mesh-external legacy --ignore-not-found namespace "istio-system" deleted namespace "bookinfo" deleted namespace "foo" deleted namespace "bar" deleted namespace "mesh-external" deleted namespace "legacy" deleted