OC_HOST=$(oc whoami --show-server) OCP_PASSWORD=$(cat "${KUBEADMIN_PASSWORD_FILE}") export OPENSHIFT_HOST=${OC_HOST} export OPENSHIFT_PASSWORD=${OCP_PASSWORD} export MULTIAZ="false" export DESTRUCTIVE="false" export NUMBER_OF_TENANTS="2" export TENANTS_CREATION_TIMEOUT="3" export OUTPUT_DIR=${ARTIFACT_DIR} # running RHOAM testsuite ./setup_external.sh Login successful. You have access to 120 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". time="2024-01-01T01:32:46Z" level=info msg="calling reset on all prometheus gauge vectors" custom_metrics=StartGaugeVector === RUN TestAPIs Running Suite: Functional Test Suite - / ======================================== Random Seed: 1704072766 Will run 35 of 61 specs ------------------------------ [BeforeSuite] /go/src/github.com/integr8ly/integreatly-operator/test/functional/suite_test.go:79 STEP: bootstrapping test environment @ 01/01/24 01:32:46.809 [BeforeSuite] PASSED [0.010 seconds] ------------------------------ integreatly AWS Specific Tests A12 - Verify CRO BlobStorage CRs Successful /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" [FAILED] in [JustBeforeEach] - /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/suite.go:863 @ 01/01/24 01:43:01.98 • [FAILED] [615.161 seconds] integreatly [JustBeforeEach] AWS Specific Tests A12 - Verify CRO BlobStorage CRs Successful [JustBeforeEach] /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:28 [It] /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 [FAILED] error waiting for RHMI CR status.stage to be "complete" In [JustBeforeEach] at: /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/suite.go:863 @ 01/01/24 01:43:01.98 ------------------------------ SS ------------------------------ integreatly managed-api Functional A25 - Verify standalone RHMI VPC exists and is configured properly /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" [FAILED] in [JustBeforeEach] - /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/suite.go:863 @ 01/01/24 01:53:17.246 • [FAILED] [615.266 seconds] integreatly [JustBeforeEach] managed-api Functional A25 - Verify standalone RHMI VPC exists and is configured properly [JustBeforeEach] /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:28 [It] /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 [FAILED] error waiting for RHMI CR status.stage to be "complete" In [JustBeforeEach] at: /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/suite.go:863 @ 01/01/24 01:53:17.246 ------------------------------ SS ------------------------------ integreatly managed-api Functional F04 - Verify AWS s3 blob storage resources exist /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" [FAILED] in [JustBeforeEach] - /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/suite.go:863 @ 01/01/24 02:03:32.471 • [FAILED] [615.225 seconds] integreatly [JustBeforeEach] managed-api Functional F04 - Verify AWS s3 blob storage resources exist [JustBeforeEach] /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:28 [It] /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 [FAILED] error waiting for RHMI CR status.stage to be "complete" In [JustBeforeEach] at: /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/suite.go:863 @ 01/01/24 02:03:32.471 ------------------------------ integreatly managed-api HAPPY PATH A01 - Verify that all stages in the integreatly-operator CR report completed /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" • [353.383 seconds] ------------------------------ integreatly managed-api HAPPY PATH A03 - Verify all namespaces have been created with the correct name /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.446 seconds] ------------------------------ integreatly managed-api HAPPY PATH A05 - Verify product operator version /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.309 seconds] ------------------------------ integreatly managed-api HAPPY PATH A07 - Verify product versions /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.296 seconds] ------------------------------ integreatly managed-api HAPPY PATH A08 - Verify all products routes are created /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.455 seconds] ------------------------------ integreatly managed-api HAPPY PATH A09 - Verify Subscription Install Plan Strategy /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.485 seconds] ------------------------------ integreatly managed-api HAPPY PATH A10 - Verify CRO Postgres CRs Successful /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.364 seconds] ------------------------------ integreatly managed-api HAPPY PATH A11 - Verify CRO Redis CRs Successful /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.373 seconds] ------------------------------ integreatly managed-api HAPPY PATH A13 - Verify Deployment resources have the expected replicas /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.634 seconds] ------------------------------ integreatly managed-api HAPPY PATH A14 - Verify Deployment Config resources have the expected replicas /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 2024-01-01T02:09:38Z INFO KubeAPIWarningLogger apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ • [1.617 seconds] ------------------------------ integreatly managed-api HAPPY PATH A15 - Verify Stateful Set resources have the expected replicas /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.358 seconds] ------------------------------ integreatly managed-api HAPPY PATH A26 - Verify Sendgrid Credentials Are Configured Properly /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.526 seconds] ------------------------------ integreatly managed-api HAPPY PATH A27 + A28 - Verify pod priority class is created and set /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 2024-01-01T02:09:43Z INFO KubeAPIWarningLogger apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ • [1.661 seconds] ------------------------------ integreatly managed-api HAPPY PATH A32 - Validate SSO config /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 Checking requested resources present in Keycloak pods Checking the link for admin is available Checking the link for dedicated admin is available A32 - Validate SSO config test succeeded • [1.469 seconds] ------------------------------ integreatly managed-api HAPPY PATH C01 - Verify Alerts are not pending or firing apart from DeadMansSwitch /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 Pattern ignored: soAvailability[a-z0-9]+ErrorBudgetBurn Pattern ignored: soAvailability[a-z0-9]+ErrorBudgetBurn Firing/Pending alert to be ignored: RHOAMUserSsoAvailability2hto1dErrorBudgetBurn Pattern ignored: soAvailability[a-z0-9]+ErrorBudgetBurn Firing/Pending alert to be ignored: RHOAMUserSsoAvailability6hto3dErrorBudgetBurn Pattern ignored: soAvailability[a-z0-9]+ErrorBudgetBurn Firing/Pending alert to be ignored: RHOAMRhssoAvailability2hto1dErrorBudgetBurn Waiting 1 minute for alerts to normalise before retrying Pattern ignored: soAvailability[a-z0-9]+ErrorBudgetBurn Firing/Pending alert to be ignored: RHOAMUserSsoAvailability2hto1dErrorBudgetBurn Pattern ignored: soAvailability[a-z0-9]+ErrorBudgetBurn Firing/Pending alert to be ignored: RHOAMUserSsoAvailability6hto3dErrorBudgetBurn Pattern ignored: soAvailability[a-z0-9]+ErrorBudgetBurn Firing/Pending alert to be ignored: RHOAMRhssoAvailability2hto1dErrorBudgetBurn • [121.469 seconds] ------------------------------ integreatly managed-api HAPPY PATH E10 - Verify Customer Grafana Route is accessible /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.459 seconds] ------------------------------ integreatly managed-api HAPPY PATH F02 - Verify PodDisruptionBudgets exist /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.329 seconds] ------------------------------ SSSS ------------------------------ integreatly managed-api IDP BASED B01B - Verify users can login to products /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 env var USER_NUMBERS was not set, defaulting to 1 Performing OpenShift HTTP client setup with URL https://console-openshift-console.apps.coexist-40ad.o2sz.s1.devshift.org/auth/login Error clicking IDP link Error while checking IDP is setup, retrying: error occurred during oauth login: OpenshiftClientSubmitForm(): failed to click testing-idp identity provider in oauth proxy login, ensure the identity provider exists on the cluster: Element not found matching expr 'a:contains('testing-idp')'. Performing OpenShift HTTP client setup with URL https://console-openshift-console.apps.coexist-40ad.o2sz.s1.devshift.org/auth/login Error clicking IDP link Error while checking IDP is setup, retrying: error occurred during oauth login: OpenshiftClientSubmitForm(): failed to click testing-idp identity provider in oauth proxy login, ensure the identity provider exists on the cluster: Element not found matching expr 'a:contains('testing-idp')'. Performing OpenShift HTTP client setup with URL https://console-openshift-console.apps.coexist-40ad.o2sz.s1.devshift.org/auth/login Error clicking IDP link Error while checking IDP is setup, retrying: error occurred during oauth login: OpenshiftClientSubmitForm(): failed to click testing-idp identity provider in oauth proxy login, ensure the identity provider exists on the cluster: Element not found matching expr 'a:contains('testing-idp')'. Performing OpenShift HTTP client setup with URL https://console-openshift-console.apps.coexist-40ad.o2sz.s1.devshift.org/auth/login Error clicking IDP link Error while checking IDP is setup, retrying: error occurred during oauth login: OpenshiftClientSubmitForm(): failed to click testing-idp identity provider in oauth proxy login, ensure the identity provider exists on the cluster: Element not found matching expr 'a:contains('testing-idp')'. Performing OpenShift HTTP client setup with URL https://console-openshift-console.apps.coexist-40ad.o2sz.s1.devshift.org/auth/login Error clicking IDP link Error while checking IDP is setup, retrying: error occurred during oauth login: OpenshiftClientSubmitForm(): failed to click testing-idp identity provider in oauth proxy login, ensure the identity provider exists on the cluster: Element not found matching expr 'a:contains('testing-idp')'. Performing OpenShift HTTP client setup with URL https://console-openshift-console.apps.coexist-40ad.o2sz.s1.devshift.org/auth/login Error clicking IDP link Error while checking IDP is setup, retrying: error occurred during oauth login: OpenshiftClientSubmitForm(): failed to click testing-idp identity provider in oauth proxy login, ensure the identity provider exists on the cluster: Element not found matching expr 'a:contains('testing-idp')'. Performing OpenShift HTTP client setup with URL https://console-openshift-console.apps.coexist-40ad.o2sz.s1.devshift.org/auth/login Trying to log into Customer Grafana: https://grafana-route-redhat-rhoam-customer-monitoring.apps.coexist-40ad.o2sz.s1.devshift.org for user: test-user01 [FAILED] in [It] - /go/src/github.com/integr8ly/integreatly-operator/test/common/product_login.go:58 @ 01/01/24 02:13:05.286 • [FAILED] [76.180 seconds] integreatly managed-api IDP BASED [It] B01B - Verify users can login to products /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 [FAILED] ChromeDP test failed with error: chrome failed to start: The setuid sandbox is not running as root. Common causes: * An unprivileged process using ptrace on it, like a debugger. * A parent process set prctl(PR_SET_NO_NEW_PRIVS, ...) Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted [0101/021303.355049:FATAL:zygote_host_impl_linux.cc(201)] Check failed: . : No such file or directory (2) [0101/021303.361740:ERROR:file_io_posix.cc(145)] open /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq: No such file or directory (2) [0101/021303.361815:ERROR:file_io_posix.cc(145)] open /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq: No such file or directory (2) [0101/021303.364205:ERROR:directory_reader_posix.cc(42)] opendir /tmp/Crashpad/attachments/c42552fb-e879-4c2a-a06b-964e39f6c069: No such file or directory (2) In [It] at: /go/src/github.com/integr8ly/integreatly-operator/test/common/product_login.go:58 @ 01/01/24 02:13:05.286 ------------------------------ integreatly managed-api IDP BASED B03 - Verify RHMI Developer User Permissions are Correct /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 not creating IDP, it's already created retrieved console master URL console-openshift-console.apps.coexist-40ad.o2sz.s1.devshift.org retrieved openshift-Oauth route Performing OpenShift HTTP client setup with URL https://console-openshift-console.apps.coexist-40ad.o2sz.s1.devshift.org/auth/login retrieved rhmi developer user tokens testing managed api developer projects Verifying RHMI Developer permissions for RHMI Resource • [2.544 seconds] ------------------------------ integreatly managed-api IDP BASED B04 - Verify Dedicated Admin User Permissions are Correct /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 not creating IDP, it's already created Performing OpenShift HTTP client setup with URL https://console-openshift-console.apps.coexist-40ad.o2sz.s1.devshift.org/auth/login Verifying Dedicated admin permissions for Secrets Resource • [2.380 seconds] ------------------------------ integreatly managed-api IDP BASED B06 - Verify users with no email get default email /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 not creating IDP, it's already created • [17.701 seconds] ------------------------------ SSS ------------------------------ integreatly managed-api IDP BASED H03 - Verify 3scale CRUDL permissions /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 RHMI CR status.stage is: "installation". Waiting for: "complete" RHMI CR status.stage is: "installation". Waiting for: "complete" STEP: Ensure testing IDP is configured @ 01/01/24 02:13:51.255 not creating IDP, it's already created STEP: Get RHMI CR @ 01/01/24 02:13:51.269 STEP: Get 3Scale URL and 3Scale clint @ 01/01/24 02:13:51.283 STEP: Login to 3Scale and become admin @ 01/01/24 02:13:51.283 STEP: Login to 3Scale @ 01/01/24 02:13:51.283 Attempting to open threescale login page with url: https://3scale-admin.apps.coexist-40ad.o2sz.s1.devshift.org/p/login STEP: Open 3Scale login page @ 01/01/24 02:13:51.283 STEP: Check if user authenticated @ 01/01/24 02:13:51.361 User is not authenticated yet, going to authenticate through rhsso url https://3scale-admin.apps.coexist-40ad.o2sz.s1.devshift.org/p/login STEP: Authenticate through RHSSO @ 01/01/24 02:13:51.361 STEP: Check if user authenticated @ 01/01/24 02:13:51.723 User is not authenticated through rhsso yet, going to select the IDP https://oauth-openshift.apps.coexist-40ad.o2sz.s1.devshift.org/oauth/authorize?scope=user%3Afull&state=4r85HCcx3XRiacfABttoSY6ZMr3d5xWyyoXO5TXLcRI.myBMn-4joH8.3scale&response_type=code&client_id=redhat-rhoam-rhsso&redirect_uri=https%3A%2F%2Fkeycloak-edge-redhat-rhoam-rhsso.apps.coexist-40ad.o2sz.s1.devshift.org%2Fauth%2Frealms%2Fopenshift%2Fbroker%2Fopenshift-v4%2Fendpoint STEP: Select the testing IDP to authenticate through RHSSO @ 01/01/24 02:13:51.723 STEP: Check if user authenticated @ 01/01/24 02:13:51.758 User is not authenticated after selecting the IDP yet, going to submit the form to authenticate the user through rhsso https://keycloak-edge-redhat-rhoam-rhsso.apps.coexist-40ad.o2sz.s1.devshift.org/auth/realms/testing-idp/protocol/openid-connect/auth?client_id=openshift&redirect_uri=https%3A%2F%2Foauth-openshift.apps.coexist-40ad.o2sz.s1.devshift.org%2Foauth2callback%2Ftesting-idp&response_type=code&scope=openid&state=Y3NyZj1JVVpOTVNuUnJPUzBZMkNtUUpsTDFjMXFGcFRmbEtxeW1MeXJQNnJYQWY4JnRoZW49JTJGb2F1dGglMkZhdXRob3JpemUlM0ZjbGllbnRfaWQlM0RyZWRoYXQtcmhvYW0tcmhzc28lMjZpZHAlM0R0ZXN0aW5nLWlkcCUyNnJlZGlyZWN0X3VyaSUzRGh0dHBzJTI1M0ElMjUyRiUyNTJGa2V5Y2xvYWstZWRnZS1yZWRoYXQtcmhvYW0tcmhzc28uYXBwcy5jb2V4aXN0LTQwYWQubzJzei5zMS5kZXZzaGlmdC5vcmclMjUyRmF1dGglMjUyRnJlYWxtcyUyNTJGb3BlbnNoaWZ0JTI1MkZicm9rZXIlMjUyRm9wZW5zaGlmdC12NCUyNTJGZW5kcG9pbnQlMjZyZXNwb25zZV90eXBlJTNEY29kZSUyNnNjb3BlJTNEdXNlciUyNTNBZnVsbCUyNnN0YXRlJTNENHI4NUhDY3gzWFJpYWNmQUJ0dG9TWTZaTXIzZDV4V3l5b1hPNVRYTGNSSS5teUJNbi00am9IOC4zc2NhbGU%3D STEP: Submit login form for testing IDP @ 01/01/24 02:13:51.758 STEP: Check if user authenticated after submitting RHSSO login form @ 01/01/24 02:13:52.58 STEP: Ensure 3Scale is available @ 01/01/24 02:13:52.649 STEP: Create a product @ 01/01/24 02:13:52.789 STEP: Delete the product @ 01/01/24 02:13:53.339 • [25.777 seconds] ------------------------------ integreatly managed-api IDP BASED H07 - ThreeScale User Promotion /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 env var USER_NUMBERS was not set, defaulting to 1 not creating IDP, it's already created STEP: Login to 3Scale @ 01/01/24 02:13:55.059 Attempting to open threescale login page with url: https://3scale-admin.apps.coexist-40ad.o2sz.s1.devshift.org/p/login STEP: Open 3Scale login page @ 01/01/24 02:13:55.059 STEP: Check if user authenticated @ 01/01/24 02:13:55.133 User is not authenticated yet, going to authenticate through rhsso url https://3scale-admin.apps.coexist-40ad.o2sz.s1.devshift.org/p/login STEP: Authenticate through RHSSO @ 01/01/24 02:13:55.133 STEP: Check if user authenticated @ 01/01/24 02:13:55.286 User is not authenticated through rhsso yet, going to select the IDP https://oauth-openshift.apps.coexist-40ad.o2sz.s1.devshift.org/oauth/authorize?scope=user%3Afull&state=tT9RzvN0WcByVQ4L21WnB4yCc73YO9YMFYrQv6-GEvg.oNtLHlkc9Yo.3scale&response_type=code&client_id=redhat-rhoam-rhsso&redirect_uri=https%3A%2F%2Fkeycloak-edge-redhat-rhoam-rhsso.apps.coexist-40ad.o2sz.s1.devshift.org%2Fauth%2Frealms%2Fopenshift%2Fbroker%2Fopenshift-v4%2Fendpoint STEP: Select the testing IDP to authenticate through RHSSO @ 01/01/24 02:13:55.286 STEP: Check if user authenticated @ 01/01/24 02:13:55.388 User is not authenticated after selecting the IDP yet, going to submit the form to authenticate the user through rhsso https://keycloak-edge-redhat-rhoam-rhsso.apps.coexist-40ad.o2sz.s1.devshift.org/auth/realms/testing-idp/protocol/openid-connect/auth?client_id=openshift&redirect_uri=https%3A%2F%2Foauth-openshift.apps.coexist-40ad.o2sz.s1.devshift.org%2Foauth2callback%2Ftesting-idp&response_type=code&scope=openid&state=Y3NyZj1aaWx2LTdiYWIzN0xGd0g1LXNvNmw3OFNNX3F1SFo2R2JXZjdpN2gtWm5rJnRoZW49JTJGb2F1dGglMkZhdXRob3JpemUlM0ZjbGllbnRfaWQlM0RyZWRoYXQtcmhvYW0tcmhzc28lMjZpZHAlM0R0ZXN0aW5nLWlkcCUyNnJlZGlyZWN0X3VyaSUzRGh0dHBzJTI1M0ElMjUyRiUyNTJGa2V5Y2xvYWstZWRnZS1yZWRoYXQtcmhvYW0tcmhzc28uYXBwcy5jb2V4aXN0LTQwYWQubzJzei5zMS5kZXZzaGlmdC5vcmclMjUyRmF1dGglMjUyRnJlYWxtcyUyNTJGb3BlbnNoaWZ0JTI1MkZicm9rZXIlMjUyRm9wZW5zaGlmdC12NCUyNTJGZW5kcG9pbnQlMjZyZXNwb25zZV90eXBlJTNEY29kZSUyNnNjb3BlJTNEdXNlciUyNTNBZnVsbCUyNnN0YXRlJTNEdFQ5Unp2TjBXY0J5VlE0TDIxV25CNHlDYzczWU85WU1GWXJRdjYtR0V2Zy5vTnRMSGxrYzlZby4zc2NhbGU%3D STEP: Submit login form for testing IDP @ 01/01/24 02:13:55.388 STEP: Check if user authenticated after submitting RHSSO login form @ 01/01/24 02:13:56.769 Trying to log into 3scale for user: test-user01 STEP: Login to 3Scale @ 01/01/24 02:13:56.896 Attempting to open threescale login page with url: https://3scale-admin.apps.coexist-40ad.o2sz.s1.devshift.org/p/login STEP: Open 3Scale login page @ 01/01/24 02:13:56.897 STEP: Check if user authenticated @ 01/01/24 02:13:56.955 User is not authenticated yet, going to authenticate through rhsso url https://3scale-admin.apps.coexist-40ad.o2sz.s1.devshift.org/p/login STEP: Authenticate through RHSSO @ 01/01/24 02:13:56.955 STEP: Check if user authenticated @ 01/01/24 02:13:57.179 User is not authenticated through rhsso yet, going to select the IDP https://oauth-openshift.apps.coexist-40ad.o2sz.s1.devshift.org/oauth/authorize?scope=user%3Afull&state=dRV-TCVsFaxVucINKtW_XKWzL5VRD0u_-IeBVwn000A.cLrWtAxn-nI.3scale&response_type=code&client_id=redhat-rhoam-rhsso&redirect_uri=https%3A%2F%2Fkeycloak-edge-redhat-rhoam-rhsso.apps.coexist-40ad.o2sz.s1.devshift.org%2Fauth%2Frealms%2Fopenshift%2Fbroker%2Fopenshift-v4%2Fendpoint STEP: Select the testing IDP to authenticate through RHSSO @ 01/01/24 02:13:57.179 STEP: Check if user authenticated @ 01/01/24 02:13:57.21 User is not authenticated after selecting the IDP yet, going to submit the form to authenticate the user through rhsso https://keycloak-edge-redhat-rhoam-rhsso.apps.coexist-40ad.o2sz.s1.devshift.org/auth/realms/testing-idp/protocol/openid-connect/auth?client_id=openshift&redirect_uri=https%3A%2F%2Foauth-openshift.apps.coexist-40ad.o2sz.s1.devshift.org%2Foauth2callback%2Ftesting-idp&response_type=code&scope=openid&state=Y3NyZj1uTXFTQmRTWVRuTkJReWg5eG1xalNxZWluSFV4SWwydV9TajBhYXdheDlnJnRoZW49JTJGb2F1dGglMkZhdXRob3JpemUlM0ZjbGllbnRfaWQlM0RyZWRoYXQtcmhvYW0tcmhzc28lMjZpZHAlM0R0ZXN0aW5nLWlkcCUyNnJlZGlyZWN0X3VyaSUzRGh0dHBzJTI1M0ElMjUyRiUyNTJGa2V5Y2xvYWstZWRnZS1yZWRoYXQtcmhvYW0tcmhzc28uYXBwcy5jb2V4aXN0LTQwYWQubzJzei5zMS5kZXZzaGlmdC5vcmclMjUyRmF1dGglMjUyRnJlYWxtcyUyNTJGb3BlbnNoaWZ0JTI1MkZicm9rZXIlMjUyRm9wZW5zaGlmdC12NCUyNTJGZW5kcG9pbnQlMjZyZXNwb25zZV90eXBlJTNEY29kZSUyNnNjb3BlJTNEdXNlciUyNTNBZnVsbCUyNnN0YXRlJTNEZFJWLVRDVnNGYXhWdWNJTkt0V19YS1d6TDVWUkQwdV8tSWVCVnduMDAwQS5jTHJXdEF4bi1uSS4zc2NhbGU%3D STEP: Submit login form for testing IDP @ 01/01/24 02:13:57.21 STEP: Check if user authenticated after submitting RHSSO login form @ 01/01/24 02:13:58.271 • [355.269 seconds] ------------------------------ SS ------------------------------ integreatly managed-api IDP BASED H34 - Verify 3scale custom SMTP full config /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 time="2024-01-01T02:19:50Z" level=info msg="operator is hive managed" [SKIPPED] in [It] - /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/node.go:463 @ 01/01/24 02:19:50.38 S [SKIPPED] [1.422 seconds] integreatly managed-api IDP BASED [It] H34 - Verify 3scale custom SMTP full config /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 [SKIPPED] Addon custom smtp values are not fully configured. This test is not ok to run. In [It] at: /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/node.go:463 @ 01/01/24 02:19:50.38 ------------------------------ integreatly managed-api IDP BASED H35 - Verify 3scale custom SMTP partial config /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 time="2024-01-01T02:19:51Z" level=info msg="operator is hive managed" [SKIPPED] in [It] - /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/node.go:463 @ 01/01/24 02:19:51.749 S [SKIPPED] [1.369 seconds] integreatly managed-api IDP BASED [It] H35 - Verify 3scale custom SMTP partial config /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 [SKIPPED] Addon custom smtp values are not partial configured. This test is not ok to run. In [It] at: /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/node.go:463 @ 01/01/24 02:19:51.749 ------------------------------ S ------------------------------ integreatly managed-api OBSERVABILITY TESTS A06 - Verify PVC /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.319 seconds] ------------------------------ integreatly managed-api OBSERVABILITY TESTS C04 - Verify Alerts exist /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.539 seconds] ------------------------------ integreatly managed-api OBSERVABILITY TESTS C08B - Verify alert links to SOPs /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 [SKIPPED] in [It] - /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/node.go:463 @ 01/01/24 02:19:55.901 S [SKIPPED] [1.294 seconds] integreatly managed-api OBSERVABILITY TESTS [It] C08B - Verify alert links to SOPs /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 [SKIPPED] Gitlab token not provided, use GITLAB_TOKEN environment variable to specify it In [It] at: /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/node.go:463 @ 01/01/24 02:19:55.901 ------------------------------ integreatly managed-api OBSERVABILITY TESTS C10B - Verify Prometheus blackbox targets /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.486 seconds] ------------------------------ integreatly managed-api OBSERVABILITY TESTS E09 - Verify customer dashboards exist /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.473 seconds] ------------------------------ integreatly managed-api OBSERVABILITY TESTS M02B - Verify RHOAM version metric is exposed in Prometheus /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 • [1.445 seconds] ------------------------------ SSSSSSS ------------------------------ integreatly managed-api SCALABILITY TESTS F05 - Verify Replicas Scale correctly in Threescale /go/src/github.com/integr8ly/integreatly-operator/test/functional/integreatly_test.go:139 Checking correct number of replicas are set Checking correct number of updated replicas are set Checking correct number of updated replicas are set Checking correct number of replicas has been reset 2024-01-01T02:21:41Z INFO KubeAPIWarningLogger apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ system-app replicas ready 2, expected 3, unavailable replicas 1 system-app replicas ready 2, expected 3, unavailable replicas 1 zync replicas ready 2, expected 3, unavailable replicas 1 • [161.844 seconds] ------------------------------ SSSSS ------------------------------ [AfterSuite] /go/src/github.com/integr8ly/integreatly-operator/test/functional/suite_test.go:108 STEP: tearing down the test environment @ 01/01/24 02:22:42.148 [AfterSuite] PASSED [0.001 seconds] ------------------------------ [ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report autogenerated by Ginkgo [ReportAfterSuite] PASSED [0.004 seconds] ------------------------------ Summarizing 4 Failures: [FAIL] integreatly [JustBeforeEach] AWS Specific Tests A12 - Verify CRO BlobStorage CRs Successful /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/suite.go:863 [FAIL] integreatly [JustBeforeEach] managed-api Functional A25 - Verify standalone RHMI VPC exists and is configured properly /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/suite.go:863 [FAIL] integreatly [JustBeforeEach] managed-api Functional F04 - Verify AWS s3 blob storage resources exist /go/pkg/mod/github.com/onsi/ginkgo/v2@v2.9.1/internal/suite.go:863 [FAIL] integreatly managed-api IDP BASED [It] B01B - Verify users can login to products /go/src/github.com/integr8ly/integreatly-operator/test/common/product_login.go:58 Ran 32 of 61 Specs in 2995.340 seconds FAIL! -- 28 Passed | 4 Failed | 0 Pending | 29 Skipped You're using deprecated Ginkgo functionality: ============================================= --ginkgo.progress is deprecated . The functionality provided by --progress was confusing and is no longer needed. Use --show-node-events instead to see node entry and exit events included in the timeline of failed and verbose specs. Or you can run with -vv to always see all node events. Lastly, --poll-progress-after and the PollProgressAfter decorator now provide a better mechanism for debugging specs that tend to get stuck. To silence deprecations that can be silenced set the following environment variable: ACK_GINKGO_DEPRECATIONS=2.9.1 --- FAIL: TestAPIs (2995.95s) FAIL