OCM_ENV=$API_HOST SET_ENVIRONMENT="1" OC_HOST=$(oc whoami --show-server) CLUSTER_ID=$(cat "${SHARED_DIR}/cluster-id") CLUSTER_NAME=$(cat "${SHARED_DIR}/cluster-name") OCM_TOKEN=$(cat /var/run/secrets/ci.openshift.io/cluster-profile/ocm-token) ROBOT_EXTRA_ARGS="-i $TEST_MARKER -e AutomationBug -e Resources-GPU -e Resources-2GPUS" RUN_SCRIPT_ARGS="--skip-oclogin true --set-urls-variables true --test-artifact-dir ${ARTIFACT_DIR}/results" export OCM_ENV export SET_ENVIRONMENT export OC_HOST export CLUSTER_NAME export OCM_TOKEN export ROBOT_EXTRA_ARGS export RUN_SCRIPT_ARGS export CLUSTER_ID mkdir "$ARTIFACT_DIR/results" # running RHOAI testsuite ./ods_ci/build/run.sh RUN SCRIPT ARGS: --skip-oclogin true --set-urls-variables true --test-artifact-dir /logs/artifacts/results ROBOT EXTRA ARGS: -i Smoke -e AutomationBug -e Resources-GPU -e Resources-2GPUS SET TEST ENVIRONMENT: 1 -- USE OCM to install IDPs: 1 -----| SET_ENVIRONMENT option is enabled. ODS-CI is going to configure the test environment for you..|----- ---> Performing log in OCM ocm envirnment: production Stage) validating user_config.json user_config.json found! Starting json validation.. --> Going through requested users configuration validating prefix: htp-user- --> suffix type: incremental_with_rand_base validating prefix: htp-basic-user- --> suffix type: incremental_with_rand_base --> Going through requested users configuration validating prefix: ldap-op- --> suffix type: incremental_with_rand_base validating prefix: ldap-usr- --> suffix type: incremental_with_rand_base validating prefix: ldap-noaccess- --> suffix type: incremental_with_rand_base validating prefix: ldap-special --> suffix type: custom Stage) Looking for LDAP and HTPASSWD already present in the cluster... ---- | Installing the required IDPs | ---- host: https://api.rhoai-9802.fkd8.p2.openshiftapps.com:6443 Stage) Setting HTPASSWD Identity provider --> Generating users based on requested configuration elaborating prefix: htp-user- --> suffix type: incremental_with_rand_base elaborating prefix: htp-basic-user- --> suffix type: incremental_with_rand_base --> Configuring HTP IDP and users Cluster name is rhoai-9802 Configuring IDP for cluster 'rhoai-9802' Identity Provider 'htpasswd' has been created. You need to ensure that there is a list of cluster administrators defined. See 'ocm create user --help' for more information. To login into the console, open https://console-openshift-console.apps.rhoai-9802.fkd8.p2.openshiftapps.com and click on htpasswd. Securely store your username and password. If you lose these credentials, you will have to delete and recreate the IDP. Added 'cluster-admins' user 'htp-user-YclmEIjrysQezbfLNixt' to cluster 'rhoai-9802' ----> Performing log in with newly created HTP user ----> Performing log in the cluster using oc CLI command Login failed (401 Unauthorized) Verify you have provided the correct credentials. Login failed 1 times. Trying again in 30 seconds (max 7 times). Login failed (401 Unauthorized) Verify you have provided the correct credentials. Login failed 2 times. Trying again in 30 seconds (max 7 times). Login failed (401 Unauthorized) Verify you have provided the correct credentials. Login failed 3 times. Trying again in 30 seconds (max 7 times). Login successful. You have access to 115 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Login successful. You have access to 115 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Login successful. You have access to 115 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Login successful. You have access to 115 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". ---> Adding additional HTP users, if needed per requested configuration Adding password for user htp-basic-user-vdzUOarZPKAowsFphGcW1 Adding password for user htp-basic-user-vdzUOarZPKAowsFphGcW2 secret/htpasswd-secret-27nqlh0o3lrov4ebi6ls2uok0mdcb37t replaced Stage) Setting LDAP Identity provider --> Generating users based on requested configuration elaborating prefix: ldap-op- --> suffix type: incremental_with_rand_base elaborating prefix: ldap-usr- --> suffix type: incremental_with_rand_base elaborating prefix: ldap-noaccess- --> suffix type: incremental_with_rand_base elaborating prefix: ldap-special --> suffix type: custom --> configuring LDAP server and users namespace/openldap created secret/openldap created service/openldap created deployment.apps/openldap created { "kind": "IdentityProvider", "type": "LDAPIdentityProvider", "href": "/api/clusters_mgmt/v1/clusters/27np95aa92tve9vb3ebrhq6u42jg3aq4/identity_providers/27nqmbuqqghau9gfkbc8819n6f7b8km1", "id": "27nqmbuqqghau9gfkbc8819n6f7b8km1", "name": "ldap-provider-qe", "mapping_method": "claim", "ldap": { "attributes": { "id": [ "dn" ], "name": [ "cn" ], "preferred_username": [ "uid" ] }, "bind_dn": "cn=admin,dc=example,dc=org", "insecure": true, "url": "ldap://openldap.openldap.svc.cluster.local:1389/dc=example,dc=org?uid" } } Stage) Configure RHODS test user groups group.user.openshift.io/rhods-admins created group.user.openshift.io/rhods-users created group.user.openshift.io/rhods-noaccess created Error from server (AlreadyExists): groups.user.openshift.io "dedicated-admins" already exists rhods-admins dedicated-admins group.user.openshift.io/rhods-admins added: "ldap-op-BGPClOYbDwKjTefHVnhp1" group.user.openshift.io/rhods-admins added: "ldap-op-BGPClOYbDwKjTefHVnhp2" group.user.openshift.io/rhods-admins added: "ldap-op-BGPClOYbDwKjTefHVnhp3" group.user.openshift.io/rhods-admins added: "ldap-op-BGPClOYbDwKjTefHVnhp4" group.user.openshift.io/rhods-admins added: "ldap-op-BGPClOYbDwKjTefHVnhp5" Added 'dedicated-admins' user 'ldap-op-BGPClOYbDwKjTefHVnhp1' to cluster 'rhoai-9802' Added 'dedicated-admins' user 'ldap-op-BGPClOYbDwKjTefHVnhp2' to cluster 'rhoai-9802' Added 'dedicated-admins' user 'ldap-op-BGPClOYbDwKjTefHVnhp3' to cluster 'rhoai-9802' Added 'dedicated-admins' user 'ldap-op-BGPClOYbDwKjTefHVnhp4' to cluster 'rhoai-9802' Added 'dedicated-admins' user 'ldap-op-BGPClOYbDwKjTefHVnhp5' to cluster 'rhoai-9802' rhods-users group.user.openshift.io/rhods-users added: "ldap-usr-awkOqWgJiMSfAHRGbdrc1" group.user.openshift.io/rhods-users added: "ldap-usr-awkOqWgJiMSfAHRGbdrc2" group.user.openshift.io/rhods-users added: "ldap-usr-awkOqWgJiMSfAHRGbdrc3" group.user.openshift.io/rhods-users added: "ldap-usr-awkOqWgJiMSfAHRGbdrc4" group.user.openshift.io/rhods-users added: "ldap-usr-awkOqWgJiMSfAHRGbdrc5" rhods-noaccess group.user.openshift.io/rhods-noaccess added: "ldap-noaccess-TMkVRFutmUEeaqnXrGKj1" group.user.openshift.io/rhods-noaccess added: "ldap-noaccess-TMkVRFutmUEeaqnXrGKj2" group.user.openshift.io/rhods-noaccess added: "ldap-noaccess-TMkVRFutmUEeaqnXrGKj3" group.user.openshift.io/rhods-noaccess added: "ldap-noaccess-TMkVRFutmUEeaqnXrGKj4" group.user.openshift.io/rhods-noaccess added: "ldap-noaccess-TMkVRFutmUEeaqnXrGKj5" rhods-users group.user.openshift.io/rhods-users added: "ldap-special." group.user.openshift.io/rhods-users added: "ldap-special^" group.user.openshift.io/rhods-users added: "ldap-special$" group.user.openshift.io/rhods-users added: "ldap-special*" group.user.openshift.io/rhods-users added: "ldap-special?" group.user.openshift.io/rhods-users added: "ldap-special(" group.user.openshift.io/rhods-users added: "ldap-special)" group.user.openshift.io/rhods-users added: "ldap-special[" group.user.openshift.io/rhods-users added: "ldap-special]" group.user.openshift.io/rhods-users added: "ldap-special{" group.user.openshift.io/rhods-users added: "ldap-special}" group.user.openshift.io/rhods-users added: "ldap-special|" group.user.openshift.io/rhods-users added: "ldap-special@" group.user.openshift.io/rhods-users added: "ldap-special;" Stage) Sleeping 180sec to wait for IDPs to become available -----| ODS-CI is starting the tests run...|----- ods_ci/test-variables.yml INFO: getting RHODS URLs from the cluster as per --set-urls-variables OCP Console URL set to: https://console-openshift-console.apps.rhoai-9802.fkd8.p2.openshiftapps.com RHODS Dashboard URL set to: https://rhods-dashboard-redhat-ods-applications.apps.rhoai-9802.fkd8.p2.openshiftapps.com RHODS API Server URL set to: https://api.rhoai-9802.fkd8.p2.openshiftapps.com:6443 RHODS Prometheus URL set to: https://prometheus-redhat-ods-monitoring.apps.rhoai-9802.fkd8.p2.openshiftapps.com INFO: we found a yq executable skipping OC login as per parameter --skip-oclogin ============================================================================== Tests ============================================================================== Tests.Deploy ============================================================================== Tests.Deploy.Installation ============================================================================== 2023-11-25 13:43:18,328 - RPA.core.certificates - INFO - Truststore not in use, HTTPS traffic validated against `certifi` package. (requires Python 3.10.12 and 'pip' 23.2.1 at minimum) Tests.Deploy.Installation.Post Install :: Post install test cases that main... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Verify RHODS Display Name and Version :: Verify consistent rhods d... | PASS | ------------------------------------------------------------------------------ Verify RHODS Notebooks Network Policies :: Verifies that the netwo... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] Executing keyword 'Wait Until Page Contains' failed: Text 'Log in with OpenShift' did not appear in 15 seconds. | PASS | ------------------------------------------------------------------------------ Verify All The Pods Are Using Image Digest Instead Of Tags :: Veri... | PASS | ------------------------------------------------------------------------------ Tests.Deploy.Installation.Post Install :: Post install test cases ... | PASS | 3 tests, 3 passed, 0 failed ============================================================================== Tests.Deploy.Installation | PASS | 3 tests, 3 passed, 0 failed ============================================================================== Tests.Deploy | PASS | 3 tests, 3 passed, 0 failed ============================================================================== Tests.Monitor And Manage ============================================================================== Tests.Monitor And Manage.Metrics ============================================================================== Tests.Monitor And Manage.Metrics.Metrics :: Test suite testing ODS Metrics ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Test Existence of Prometheus Alerting Rules :: Verifies the promet... | FAIL | SLOs-haproxy_backend_http_responses_dsp was not found in Prometheus rules ------------------------------------------------------------------------------ Test Existence of Prometheus Recording Rules :: Verifies the prome... | FAIL | SLOs - RHODS Operator v2 was not found in Prometheus rules ------------------------------------------------------------------------------ Tests.Monitor And Manage.Metrics.Metrics :: Test suite testing ODS... | FAIL | 2 tests, 0 passed, 2 failed ============================================================================== Tests.Monitor And Manage.Metrics.Billing Metrics :: Test suite testing ODS ... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Verify OpenShift Monitoring Results Are Correct When Running Undef... [ WARN ] Exception managing chrome: Permission denied (os error 13) | FAIL | Element 'xpath=//*[@data-test-id="perspective-switcher-toggle"]/span/h2' did not appear in 30 seconds. ------------------------------------------------------------------------------ Test Billing Metric (Notebook Cpu Usage) On OpenShift Monitoring :... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] Executing keyword 'Wait Until Page Contains' failed: Text 'Log in with OpenShift' did not appear in 15 seconds. [ WARN ] Executing keyword 'Wait Until Page Contains' failed: Text 'Cloning...' did not appear in 5 seconds. [ WARN ] Executing keyword 'Wait Until Page Contains' failed: Text 'Successfully cloned' did not appear in 1 minute. [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] Exception managing chrome: Permission denied (os error 13) | FAIL | Element 'xpath=//*[@data-test-id="perspective-switcher-toggle"]/span/h2' did not appear in 30 seconds. ------------------------------------------------------------------------------ Test Metric "Rhods_Aggregate_Availability" On Cluster Monitoring P... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Monitor And Manage.Metrics.Billing Metrics :: Test suite tes... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== Tests.Monitor And Manage.Metrics | FAIL | 5 tests, 1 passed, 4 failed ============================================================================== Tests.Monitor And Manage.Alerts ============================================================================== Tests.Monitor And Manage.Alerts.Alerts :: RHODS monitoring alerts test suite ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Verify All Alerts Severity :: Verifies that all alerts have the ex... [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 120" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 900" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 3600" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 10800" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Jupyter Probe Success Burn Rate 120" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Jupyter Probe Success Burn Rate 900" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Jupyter Probe Success Burn Rate 3600" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Jupyter Probe Success Burn Rate 10800" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Probe Success Burn Rate 120" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Probe Success Burn Rate 900" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Probe Success Burn Rate 3600" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Probe Success Burn Rate 10800" was not found in Prometheus | FAIL | Several failures occurred: 1) Alert "RHODS Dashboard Route Error Burn Rate 120" was not found in Prometheus 2) Alert "RHODS Dashboard Route Error Burn Rate 900" was not found in Prometheus 3) Alert "RHODS Dashboard Route Error Burn Rate 3600" was not found in Prometheus 4) Alert "RHODS Dashboard Route Error Burn Rate 10800" was not found in Prometheus 5) Alert "RHODS Jupyter Probe Success Burn Rate 120" was not found in Prometheus 6) Alert "RHODS Jupyter Probe Success Burn Rate 900" was not found in Prometheus 7) Alert "RHODS Jupyter Probe Success Burn Rate 3600" was not found in Prometheus 8) Alert "RHODS Jupyter Probe Success Burn Rate 10800" was not found in Prometheus 9) Alert "RHODS Dashboard Probe Success Burn Rate 120" was not found in Prometheus 10) Alert "RHODS Dashboard Probe Success Burn Rate 900" was not found in Prometheus 11) Alert "RHODS Dashboard Probe Success Burn Rate 3600" was not found in Prometheus 12) Alert "RHODS Dashboard Probe Success Burn Rate 10800" was not found in Prometheus ------------------------------------------------------------------------------ Verify No Alerts Are Firing Except For DeadManSnitch :: Verifies t... [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 120" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 900" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 3600" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 10800" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Jupyter Probe Success Burn Rate 120" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Jupyter Probe Success Burn Rate 900" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Jupyter Probe Success Burn Rate 3600" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Jupyter Probe Success Burn Rate 10800" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Probe Success Burn Rate 120" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Probe Success Burn Rate 900" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Probe Success Burn Rate 3600" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Probe Success Burn Rate 10800" was not found in Prometheus | PASS | ------------------------------------------------------------------------------ Tests.Monitor And Manage.Alerts.Alerts :: RHODS monitoring alerts ... | FAIL | 2 tests, 1 passed, 1 failed ============================================================================== Tests.Monitor And Manage.Alerts | FAIL | 2 tests, 1 passed, 1 failed ============================================================================== Tests.Monitor And Manage | FAIL | 7 tests, 2 passed, 5 failed ============================================================================== Tests.Ods Dashboard ============================================================================== Tests.Ods Dashboard.Ods Dashboard ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Verify RHODS Explore Section Contains Only Expected ISVs :: It ver... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Verify CSS Style Of Getting Started Descriptions :: Verifies the C... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Ods Dashboard.Ods Dashboard | PASS | 2 tests, 2 passed, 0 failed ============================================================================== Tests.Ods Dashboard.Ods Dashboard Settings :: Tests features in ODS Dashboa... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Verify That Administrators Can Access "Cluster Settings" :: Verifi... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Ods Dashboard.Ods Dashboard Settings :: Tests features in OD... | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ods Dashboard.Ods Dashboard Projects ============================================================================== Tests.Ods Dashboard.Ods Dashboard Projects.Ods Dashboard Projects :: Suite ... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Exception managing chrome: Permission denied (os error 13) Verify Data Science Projects Page Is Accessible :: Verifies "Data ... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Verify User Can Access Jupyter Launcher From DS Project Page :: Ve... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Verify Workbench Images Have Multiple Versions :: Verifies that wo... [ WARN ] Exception managing chrome: Permission denied (os error 13) | FAIL | Element 'xpath=//section[@id="notebook-image"]//div[@id="workbench-image-stream-selection"]/button' did not appear in 5 seconds. ------------------------------------------------------------------------------ Verify User Can Create A Data Science Project :: Verifies users ca... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Verify User Can Create And Start A Workbench With Existent PV Stor... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] msg=There is probably no Workbench with Diplay Name equal to ODS-CI Workbench 2 | FAIL | Element 'xpath=//section[@id="notebook-image"]//div[@id="workbench-image-stream-selection"]/button' did not appear in 5 seconds. ------------------------------------------------------------------------------ Verify User Can Create And Start A Workbench Adding A New PV Stora... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] msg=There is probably no Workbench with Diplay Name equal to ODS-CI Workbench 3 | FAIL | Element 'xpath=//section[@id="notebook-image"]//div[@id="workbench-image-stream-selection"]/button' did not appear in 5 seconds. Also teardown failed: ResourceOperationFailed: Delete failed Reason: Src or at least one of name, label_selector or field_selector is required ------------------------------------------------------------------------------ Verify Users Can Start, Stop, Launch And Delete A Workbench :: Ver... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] msg=There is probably no Workbench with Diplay Name equal to ODS-CI Workbench 1 | FAIL | Setup failed: Element 'xpath=//section[@id="notebook-image"]//div[@id="workbench-image-stream-selection"]/button' did not appear in 5 seconds. Also teardown failed: ResourceOperationFailed: Delete failed Reason: Src or at least one of name, label_selector or field_selector is required ------------------------------------------------------------------------------ Verify Users Can Start, Stop And Launch A Workbench From DS Projec... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] msg=There is probably no Workbench with Diplay Name equal to ODS-CI Workbench 1 | FAIL | Setup failed: Element 'xpath=//section[@id="notebook-image"]//div[@id="workbench-image-stream-selection"]/button' did not appear in 5 seconds. Also teardown failed: ResourceOperationFailed: Delete failed Reason: Src or at least one of name, label_selector or field_selector is required ------------------------------------------------------------------------------ Verify User Can Delete A Data Science Project :: Verifies users ca... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] No Modals on the screen right now.. | PASS | ------------------------------------------------------------------------------ [ WARN ] msg=There is probably no DS Projects with Diplay Name equal to ODS-CI Common Prj Tests.Ods Dashboard.Ods Dashboard Projects.Ods Dashboard Projects ... | FAIL | 9 tests, 4 passed, 5 failed ============================================================================== Tests.Ods Dashboard.Ods Dashboard Projects.Ods Dashboard Projects Permissio... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] Exception managing chrome: Permission denied (os error 13) Verify User Can Access Permission Tab In Their Owned DS Project ::... | PASS | The Test is executed as part of Suite Setup ------------------------------------------------------------------------------ Tests.Ods Dashboard.Ods Dashboard Projects.Ods Dashboard Projects ... | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ods Dashboard.Ods Dashboard Projects | FAIL | 10 tests, 5 passed, 5 failed ============================================================================== Tests.Ods Dashboard.Model Serving ============================================================================== Tests.Ods Dashboard.Model Serving.Model Serving ============================================================================== error: the server doesn't have a resource type "datasciencecluster" Verify Model Serving Installation :: Verifies that the core compon... | SKIP | Skipped in parent suite setup: "false" == "false" ------------------------------------------------------------------------------ Verify Openvino_IR Model Via UI :: Test the deployment of an openv... | SKIP | Skipped in parent suite setup: "false" == "false" ------------------------------------------------------------------------------ Test Inference Without Token Authentication :: Test the inference ... | SKIP | Skipped in parent suite setup: "false" == "false" ------------------------------------------------------------------------------ Model not deployed, skipping deletion step during teardown Tests.Ods Dashboard.Model Serving.Model Serving | SKIP | Skipped in suite setup: "false" == "false" 3 tests, 0 passed, 0 failed, 3 skipped ============================================================================== Tests.Ods Dashboard.Model Serving.Model Serving Llm :: Collection of CLI te... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring error: the server doesn't have a resource type "datasciencecluster" Verify User Can Serve And Query A Model :: Basic tests for prepari... | SKIP | Skipped in parent suite setup: "false" == "false" ------------------------------------------------------------------------------ Verify Non Admin Can Serve And Query A Model :: Basic tests levera... | SKIP | Skipped in parent suite setup: "false" == "false" ------------------------------------------------------------------------------ Verify User Can Access Model Metrics From UWM :: Verifies that mod... | SKIP | Skipped in parent suite setup: "false" == "false" ------------------------------------------------------------------------------ Verify User Can Query A Model Using HTTP Calls :: From RHOAI 2.5 H... | SKIP | Skipped in parent suite setup: "false" == "false" ------------------------------------------------------------------------------ Tests.Ods Dashboard.Model Serving.Model Serving Llm :: Collection ... | SKIP | Skipped in suite setup: "false" == "false" 4 tests, 0 passed, 0 failed, 4 skipped ============================================================================== Tests.Ods Dashboard.Model Serving.Model Serving Customruntimes :: Collectio... ============================================================================== [ WARN ] Exception managing chrome: Permission denied (os error 13) Prometheus is running in the namespace: redhat-ods-monitoring Verify RHODS Admins Can Import A Custom Serving Runtime Template B... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Verify RHODS Admins Can Delete A Custom Serving Runtime Template [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] No Modals on the screen right now.. [ WARN ] No Modals on the screen right now.. [ WARN ] No Modals on the screen right now.. | PASS | ------------------------------------------------------------------------------ [ WARN ] msg=There is probably no DS Projects with Diplay Name equal to CustomServingRuntimesProject Tests.Ods Dashboard.Model Serving.Model Serving Customruntimes :: ... | PASS | 2 tests, 2 passed, 0 failed ============================================================================== Tests.Ods Dashboard.Model Serving | PASS | 9 tests, 2 passed, 0 failed, 7 skipped ============================================================================== Tests.Ods Dashboard.Data Science Pipelines ============================================================================== Tests.Ods Dashboard.Data Science Pipelines.Data-Science-Pipelines-Ui :: Sui... ============================================================================== Installing OpenShift Pipelines operator: pipelines-1.12 (GCP 4.14 Self-Managed: False) Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Verify User Can Create, Run and Delete A DS Pipeline From DS Proje... | FAIL | Parent suite setup failed: Error installing OpenShift Pipelines operator ------------------------------------------------------------------------------ Tests.Ods Dashboard.Data Science Pipelines.Data-Science-Pipelines-... | FAIL | Suite setup failed: Error installing OpenShift Pipelines operator Also suite teardown failed: Variable '${PROJECTS_TO_DELETE}' not found. 1 test, 0 passed, 1 failed ============================================================================== Tests.Ods Dashboard.Data Science Pipelines | FAIL | 1 test, 0 passed, 1 failed ============================================================================== Tests.Ods Dashboard | FAIL | 23 tests, 10 passed, 6 failed, 7 skipped ============================================================================== Tests.Jupyterhub ============================================================================== Tests.Jupyterhub.Base-User-Shutdown-Test :: Test Case that verifies a base ... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Verify Base User Can Stop A Running Server :: Verifies that a base... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ ERROR ] Pods not found in search | PASS | ------------------------------------------------------------------------------ Tests.Jupyterhub.Base-User-Shutdown-Test :: Test Case that verifie... | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Jupyterhub.Multiple-Image-Tags :: Test Suite for the multiple noteboo... ============================================================================== Verify All OOTB Images Have Version Dropdowns :: Verifies all imag... Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Exception managing chrome: Permission denied (os error 13) | FAIL | Page should have contained element 'xpath=//input[contains(@id, "minimal-notebook")]/../../div[contains(@class, "notebook-image-tags")]//label//div[contains(string(), "2023.2")]' but did not. ------------------------------------------------------------------------------ Tests.Jupyterhub.Multiple-Image-Tags :: Test Suite for the multipl... | FAIL | 1 test, 0 passed, 1 failed ============================================================================== Tests.Jupyterhub.Special-User-Testing ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Test Special Usernames [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Jupyterhub.Special-User-Testing | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Jupyterhub.Test ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Logged Into OpenShift [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Can Launch Jupyterhub | PASS | ------------------------------------------------------------------------------ Can Login To Jupyterhub | PASS | ------------------------------------------------------------------------------ Can Spawn Notebook [ WARN ] Executing keyword 'Wait Until Page Contains' failed: Text 'Log in with OpenShift' did not appear in 15 seconds. | PASS | ------------------------------------------------------------------------------ Tests.Jupyterhub.Test | PASS | 4 tests, 4 passed, 0 failed ============================================================================== Tests.Jupyterhub | FAIL | 7 tests, 6 passed, 1 failed ============================================================================== Tests.Must Gather ============================================================================== Tests.Must Gather.Test-Must-Gather-Logs ============================================================================== Verify that the must-gather image provides RHODS logs and info | PASS | ------------------------------------------------------------------------------ Tests.Must Gather.Test-Must-Gather-Logs | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Must Gather | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps ============================================================================== Tests.Ai Apps.Anaconda Commercial Edition ============================================================================== Tests.Ai Apps.Anaconda Commercial Edition.Anaconda Commercial Edition :: Te... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Verify Anaconda Professional Is Available In RHODS Dashboard Explo... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Ai Apps.Anaconda Commercial Edition.Anaconda Commercial Edit... | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Anaconda Commercial Edition | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Ibm Watson Studio ============================================================================== Tests.Ai Apps.Ibm Watson Studio.Ibm Watson Studio ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Verify IBM Watson Studio Is Available In RHODS Dashboard Explore Page [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Ai Apps.Ibm Watson Studio.Ibm Watson Studio | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Ibm Watson Studio | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Intel Aikit ============================================================================== Tests.Ai Apps.Intel Aikit.Intel Aikit ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Verify Intel AIKIT Is Available In RHODS Dashboard Explore Page [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Ai Apps.Intel Aikit.Intel Aikit | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Intel Aikit | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Rhoam ============================================================================== Tests.Ai Apps.Rhoam.Rhoam ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Verify RHOAM Is Available In RHODS Dashboard Explore Page [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Ai Apps.Rhoam.Rhoam | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Rhoam | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Openvino ============================================================================== Tests.Ai Apps.Openvino.Openvino ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring Verify OpenVino Is Available In RHODS Dashboard Explore Page [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Ai Apps.Openvino.Openvino | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Openvino | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps | PASS | 5 tests, 5 passed, 0 failed ============================================================================== Tests | FAIL | 46 tests, 27 passed, 12 failed, 7 skipped ============================================================================== Output: /logs/artifacts/results/ods-ci-2023-11-25-13-43-ZBUyfphMwi/output.xml XUnit: /logs/artifacts/results/ods-ci-2023-11-25-13-43-ZBUyfphMwi/xunit_test_result.xml Log: /logs/artifacts/results/ods-ci-2023-11-25-13-43-ZBUyfphMwi/log.html Report: /logs/artifacts/results/ods-ci-2023-11-25-13-43-ZBUyfphMwi/test_report.html 12