OCM_ENV=$API_HOST SET_ENVIRONMENT="1" OC_HOST=$(oc whoami --show-server) CLUSTER_ID=$(cat "${SHARED_DIR}/cluster-id") CLUSTER_NAME=$(cat "${SHARED_DIR}/cluster-name") OCM_TOKEN=$(cat /var/run/secrets/ci.openshift.io/cluster-profile/ocm-token) ROBOT_EXTRA_ARGS="-i $TEST_MARKER -e AutomationBug -e Resources-GPU -e Resources-2GPUS" RUN_SCRIPT_ARGS="--skip-oclogin true --set-urls-variables true --test-artifact-dir ${ARTIFACT_DIR}/results" export OCM_ENV export SET_ENVIRONMENT export OC_HOST export CLUSTER_NAME export OCM_TOKEN export ROBOT_EXTRA_ARGS export RUN_SCRIPT_ARGS export CLUSTER_ID mkdir "$ARTIFACT_DIR/results" # running RHOAI testsuite ./ods_ci/build/run.sh RUN SCRIPT ARGS: --skip-oclogin true --set-urls-variables true --test-artifact-dir /logs/artifacts/results ROBOT EXTRA ARGS: -i Smoke -e AutomationBug -e Resources-GPU -e Resources-2GPUS SET TEST ENVIRONMENT: 1 -- USE OCM to install IDPs: 1 -----| SET_ENVIRONMENT option is enabled. ODS-CI is going to configure the test environment for you..|----- ---> Performing log in OCM ocm envirnment: stage Stage) validating user_config.json user_config.json found! Starting json validation.. --> Going through requested users configuration validating prefix: htp-user- --> suffix type: incremental_with_rand_base validating prefix: htp-basic-user- --> suffix type: incremental_with_rand_base --> Going through requested users configuration validating prefix: ldap-op- --> suffix type: incremental_with_rand_base validating prefix: ldap-usr- --> suffix type: incremental_with_rand_base validating prefix: ldap-noaccess- --> suffix type: incremental_with_rand_base validating prefix: ldap-special --> suffix type: custom Stage) Looking for LDAP and HTPASSWD already present in the cluster... ---- | Installing the required IDPs | ---- host: https://api.rhods-7242.x9aa.s2.devshift.org:6443 Stage) Setting HTPASSWD Identity provider --> Generating users based on requested configuration elaborating prefix: htp-user- --> suffix type: incremental_with_rand_base elaborating prefix: htp-basic-user- --> suffix type: incremental_with_rand_base --> Configuring HTP IDP and users Cluster name is rhods-7242 Configuring IDP for cluster 'rhods-7242' Identity Provider 'htpasswd' has been created. You need to ensure that there is a list of cluster administrators defined. See 'ocm create user --help' for more information. To login into the console, open https://console-openshift-console.apps.rhods-7242.x9aa.s2.devshift.org and click on htpasswd. Securely store your username and password. If you lose these credentials, you will have to delete and recreate the IDP. Added 'cluster-admins' user 'htp-user-vZOpYfrGzMCDXAgFiewj' to cluster 'rhods-7242' ----> Performing log in with newly created HTP user ----> Performing log in the cluster using oc CLI command Login failed (401 Unauthorized) Verify you have provided the correct credentials. Login failed 1 times. Trying again in 30 seconds (max 7 times). Login failed (401 Unauthorized) Verify you have provided the correct credentials. Login failed 2 times. Trying again in 30 seconds (max 7 times). Login successful. You have access to 116 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Login successful. You have access to 116 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Login successful. You have access to 116 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Login successful. You have access to 116 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Login successful. You have access to 116 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". ---> Adding additional HTP users, if needed per requested configuration Adding password for user htp-basic-user-WcQGjlZbUDpAzaSnTIht1 Adding password for user htp-basic-user-WcQGjlZbUDpAzaSnTIht2 secret/htpasswd-secret-27cgp1kn7rnarn04pak50sc4676r88q5 replaced Stage) Setting LDAP Identity provider --> Generating users based on requested configuration elaborating prefix: ldap-op- --> suffix type: incremental_with_rand_base elaborating prefix: ldap-usr- --> suffix type: incremental_with_rand_base elaborating prefix: ldap-noaccess- --> suffix type: incremental_with_rand_base elaborating prefix: ldap-special --> suffix type: custom --> configuring LDAP server and users namespace/openldap created secret/openldap created service/openldap created deployment.apps/openldap created { "kind": "IdentityProvider", "type": "LDAPIdentityProvider", "href": "/api/clusters_mgmt/v1/clusters/27cf7qmcpnniqh3kec8ue9e8fdg37iot/identity_providers/27cgpl10b7ircqutahft6p7rg7ibhgrq", "id": "27cgpl10b7ircqutahft6p7rg7ibhgrq", "name": "ldap-provider-qe", "mapping_method": "claim", "ldap": { "attributes": { "id": [ "dn" ], "name": [ "cn" ], "preferred_username": [ "uid" ] }, "bind_dn": "cn=admin,dc=example,dc=org", "insecure": true, "url": "ldap://openldap.openldap.svc.cluster.local:1389/dc=example,dc=org?uid" } } Stage) Configure RHODS test user groups group.user.openshift.io/rhods-admins created group.user.openshift.io/rhods-users created group.user.openshift.io/rhods-noaccess created Error from server (AlreadyExists): groups.user.openshift.io "dedicated-admins" already exists rhods-admins dedicated-admins group.user.openshift.io/rhods-admins added: "ldap-op-UkKrIayoVNhpuXQORdjD1" group.user.openshift.io/rhods-admins added: "ldap-op-UkKrIayoVNhpuXQORdjD2" group.user.openshift.io/rhods-admins added: "ldap-op-UkKrIayoVNhpuXQORdjD3" group.user.openshift.io/rhods-admins added: "ldap-op-UkKrIayoVNhpuXQORdjD4" group.user.openshift.io/rhods-admins added: "ldap-op-UkKrIayoVNhpuXQORdjD5" Added 'dedicated-admins' user 'ldap-op-UkKrIayoVNhpuXQORdjD1' to cluster 'rhods-7242' Added 'dedicated-admins' user 'ldap-op-UkKrIayoVNhpuXQORdjD2' to cluster 'rhods-7242' Added 'dedicated-admins' user 'ldap-op-UkKrIayoVNhpuXQORdjD3' to cluster 'rhods-7242' Added 'dedicated-admins' user 'ldap-op-UkKrIayoVNhpuXQORdjD4' to cluster 'rhods-7242' Added 'dedicated-admins' user 'ldap-op-UkKrIayoVNhpuXQORdjD5' to cluster 'rhods-7242' rhods-users group.user.openshift.io/rhods-users added: "ldap-usr-uKBnqLSGaVMTozfRhUCp1" group.user.openshift.io/rhods-users added: "ldap-usr-uKBnqLSGaVMTozfRhUCp2" group.user.openshift.io/rhods-users added: "ldap-usr-uKBnqLSGaVMTozfRhUCp3" group.user.openshift.io/rhods-users added: "ldap-usr-uKBnqLSGaVMTozfRhUCp4" group.user.openshift.io/rhods-users added: "ldap-usr-uKBnqLSGaVMTozfRhUCp5" rhods-noaccess group.user.openshift.io/rhods-noaccess added: "ldap-noaccess-LrsENzdtRunUeqMOxZpo1" group.user.openshift.io/rhods-noaccess added: "ldap-noaccess-LrsENzdtRunUeqMOxZpo2" group.user.openshift.io/rhods-noaccess added: "ldap-noaccess-LrsENzdtRunUeqMOxZpo3" group.user.openshift.io/rhods-noaccess added: "ldap-noaccess-LrsENzdtRunUeqMOxZpo4" group.user.openshift.io/rhods-noaccess added: "ldap-noaccess-LrsENzdtRunUeqMOxZpo5" rhods-users group.user.openshift.io/rhods-users added: "ldap-special." group.user.openshift.io/rhods-users added: "ldap-special^" group.user.openshift.io/rhods-users added: "ldap-special$" group.user.openshift.io/rhods-users added: "ldap-special*" group.user.openshift.io/rhods-users added: "ldap-special?" group.user.openshift.io/rhods-users added: "ldap-special(" group.user.openshift.io/rhods-users added: "ldap-special)" group.user.openshift.io/rhods-users added: "ldap-special[" group.user.openshift.io/rhods-users added: "ldap-special]" group.user.openshift.io/rhods-users added: "ldap-special{" group.user.openshift.io/rhods-users added: "ldap-special}" group.user.openshift.io/rhods-users added: "ldap-special|" group.user.openshift.io/rhods-users added: "ldap-special@" group.user.openshift.io/rhods-users added: "ldap-special;" Stage) Sleeping 180sec to wait for IDPs to become available -----| ODS-CI is starting the tests run...|----- ods_ci/test-variables.yml INFO: getting RHODS URLs from the cluster as per --set-urls-variables OCP Console URL set to: https://console-openshift-console.apps.rhods-7242.x9aa.s2.devshift.org RHODS Dashboard URL set to: https://rhods-dashboard-redhat-ods-applications.apps.rhods-7242.x9aa.s2.devshift.org RHODS API Server URL set to: https://api.rhods-7242.x9aa.s2.devshift.org:6443 RHODS Prometheus URL set to: https://prometheus-redhat-ods-monitoring.apps.rhods-7242.x9aa.s2.devshift.org INFO: we found a yq executable skipping OC login as per parameter --skip-oclogin ============================================================================== Tests ============================================================================== Tests.Deploy ============================================================================== Tests.Deploy.Installation ============================================================================== 2023-11-08 09:57:46,309 - RPA.core.certificates - INFO - Truststore not in use, HTTPS traffic validated against `certifi` package. (requires Python 3.10.12 and 'pip' 23.2.1 at minimum) Tests.Deploy.Installation.Post Install :: Post install test cases that main... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Deploy.Installation.Post Install: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Verify RHODS Display Name and Version :: Verify consistent rhods d... | PASS | ------------------------------------------------------------------------------ Verify RHODS Notebooks Network Policies :: Verifies that the netwo... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] Executing keyword 'Wait Until Page Contains' failed: Text 'Log in with OpenShift' did not appear in 15 seconds. | PASS | ------------------------------------------------------------------------------ Verify All The Pods Are Using Image Digest Instead Of Tags :: Veri... | PASS | ------------------------------------------------------------------------------ [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Trdwn: Tests.Deploy.Installation.Post Install: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Tests.Deploy.Installation.Post Install :: Post install test cases ... | PASS | 3 tests, 3 passed, 0 failed ============================================================================== Tests.Deploy.Installation | PASS | 3 tests, 3 passed, 0 failed ============================================================================== Tests.Deploy | PASS | 3 tests, 3 passed, 0 failed ============================================================================== Tests.Monitor And Manage ============================================================================== Tests.Monitor And Manage.Metrics ============================================================================== Tests.Monitor And Manage.Metrics.Metrics :: Test suite testing ODS Metrics ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Monitor And Manage.Metrics.Metrics: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Test Existence of Prometheus Alerting Rules :: Verifies the promet... | FAIL | SLOs-haproxy_backend_http_responses_total was not found in Prometheus rules ------------------------------------------------------------------------------ Test Existence of Prometheus Recording Rules :: Verifies the prome... | FAIL | SLOs - ODH Dashboard was not found in Prometheus rules ------------------------------------------------------------------------------ [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Trdwn: Tests.Monitor And Manage.Metrics.Metrics: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Tests.Monitor And Manage.Metrics.Metrics :: Test suite testing ODS... | FAIL | 2 tests, 0 passed, 2 failed ============================================================================== Tests.Monitor And Manage.Metrics.Billing Metrics :: Test suite testing ODS ... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Monitor And Manage.Metrics.Billing Metrics: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Verify OpenShift Monitoring Results Are Correct When Running Undef... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Test Billing Metric (Notebook Cpu Usage) On OpenShift Monitoring :... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] Executing keyword 'Wait Until Page Contains' failed: Text 'Log in with OpenShift' did not appear in 15 seconds. [ WARN ] Executing keyword 'Wait Until Page Contains' failed: Text 'Cloning...' did not appear in 5 seconds. [ WARN ] Executing keyword 'Wait Until Page Contains' failed: Text 'Successfully cloned' did not appear in 1 minute. [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Trdwn: Tests.Monitor And Manage.Metrics.Billing Metrics: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Tests.Monitor And Manage.Metrics.Billing Metrics :: Test suite tes... | PASS | 2 tests, 2 passed, 0 failed ============================================================================== Tests.Monitor And Manage.Metrics | FAIL | 4 tests, 2 passed, 2 failed ============================================================================== Tests.Monitor And Manage.Alerts ============================================================================== Tests.Monitor And Manage.Alerts.Alerts :: RHODS monitoring alerts test suite ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Monitor And Manage.Alerts.Alerts: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Verify All Alerts Severity :: Verifies that all alerts have the ex... [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 120" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 900" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 3600" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 10800" was not found in Prometheus | FAIL | Several failures occurred: 1) Alert "RHODS Dashboard Route Error Burn Rate 120" was not found in Prometheus 2) Alert "RHODS Dashboard Route Error Burn Rate 900" was not found in Prometheus 3) Alert "RHODS Dashboard Route Error Burn Rate 3600" was not found in Prometheus 4) Alert "RHODS Dashboard Route Error Burn Rate 10800" was not found in Prometheus ------------------------------------------------------------------------------ Verify No Alerts Are Firing Except For DeadManSnitch :: Verifies t... [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 120" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 900" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 3600" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Route Error Burn Rate 10800" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Probe Success Burn Rate 900" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Probe Success Burn Rate 3600" was not found in Prometheus [ WARN ] ERROR: Alert "RHODS Dashboard Probe Success Burn Rate 10800" was not found in Prometheus | PASS | ------------------------------------------------------------------------------ [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Trdwn: Tests.Monitor And Manage.Alerts.Alerts: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Tests.Monitor And Manage.Alerts.Alerts :: RHODS monitoring alerts ... | FAIL | 2 tests, 1 passed, 1 failed ============================================================================== Tests.Monitor And Manage.Alerts | FAIL | 2 tests, 1 passed, 1 failed ============================================================================== Tests.Monitor And Manage | FAIL | 6 tests, 3 passed, 3 failed ============================================================================== Tests.Ods Dashboard ============================================================================== Tests.Ods Dashboard.Ods Dashboard ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Ods Dashboard.Ods Dashboard: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Verify RHODS Explore Section Contains Only Expected ISVs :: It ver... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Verify CSS Style Of Getting Started Descriptions :: Verifies the C... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Trdwn: Tests.Ods Dashboard.Ods Dashboard: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Tests.Ods Dashboard.Ods Dashboard | PASS | 2 tests, 2 passed, 0 failed ============================================================================== Tests.Ods Dashboard.Ods Dashboard Settings :: Tests features in ODS Dashboa... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Ods Dashboard.Ods Dashboard Settings: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Verify That Administrators Can Access "Cluster Settings" :: Verifi... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Trdwn: Tests.Ods Dashboard.Ods Dashboard Settings: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Tests.Ods Dashboard.Ods Dashboard Settings :: Tests features in OD... | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ods Dashboard.Ods Dashboard Projects ============================================================================== Tests.Ods Dashboard.Ods Dashboard Projects.Ods Dashboard Projects :: Suite ... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Ods Dashboard.Ods Dashboard Projects.Ods Dashboard Projects: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] [ WARN ] Exception managing chrome: Permission denied (os error 13) Verify Data Science Projects Page Is Accessible :: Verifies "Data ... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Verify User Can Access Jupyter Launcher From DS Project Page :: Ve... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Verify Workbench Images Have Multiple Versions :: Verifies that wo... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Verify User Can Create A Data Science Project :: Verifies users ca... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Verify User Can Create And Start A Workbench With Existent PV Stor... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] No Modals on the screen right now.. | PASS | ------------------------------------------------------------------------------ Verify User Can Create And Start A Workbench Adding A New PV Stora... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] No Modals on the screen right now.. | PASS | ------------------------------------------------------------------------------ Verify Users Can Start, Stop, Launch And Delete A Workbench :: Ver... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] msg="pv_existent" argument not set, using default PV settings [ WARN ] No Modals on the screen right now.. [ WARN ] No Modals on the screen right now.. [ WARN ] No Modals on the screen right now.. [ WARN ] Executing keyword 'Wait Until Page Contains' failed: Text 'Log in with OpenShift' did not appear in 15 seconds. [ WARN ] No Modals on the screen right now.. [ WARN ] msg=There is probably no Workbench with Diplay Name equal to ODS-CI Workbench 1 | PASS | ------------------------------------------------------------------------------ Verify Users Can Start, Stop And Launch A Workbench From DS Projec... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] msg="pv_existent" argument not set, using default PV settings [ WARN ] No Modals on the screen right now.. [ WARN ] Executing keyword 'Wait Until Page Contains' failed: Text 'Log in with OpenShift' did not appear in 15 seconds. | PASS | ------------------------------------------------------------------------------ Verify User Can Delete A Data Science Project :: Verifies users ca... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] No Modals on the screen right now.. | PASS | ------------------------------------------------------------------------------ [ WARN ] msg=There is probably no DS Projects with Diplay Name equal to ODS-CI Common Prj [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Trdwn: Tests.Ods Dashboard.Ods Dashboard Projects.Ods Dashboard Projects: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Tests.Ods Dashboard.Ods Dashboard Projects.Ods Dashboard Projects ... | PASS | 9 tests, 9 passed, 0 failed ============================================================================== Tests.Ods Dashboard.Ods Dashboard Projects.Ods Dashboard Projects Permissio... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Ods Dashboard.Ods Dashboard Projects.Ods Dashboard Projects Permissions Mgmt: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] Exception managing chrome: Permission denied (os error 13) Verify User Can Access Permission Tab In Their Owned DS Project ::... | PASS | The Test is executed as part of Suite Setup ------------------------------------------------------------------------------ [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Trdwn: Tests.Ods Dashboard.Ods Dashboard Projects.Ods Dashboard Projects Permissions Mgmt: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Tests.Ods Dashboard.Ods Dashboard Projects.Ods Dashboard Projects ... | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ods Dashboard.Ods Dashboard Projects | PASS | 10 tests, 10 passed, 0 failed ============================================================================== Tests.Ods Dashboard.Model Serving ============================================================================== Tests.Ods Dashboard.Model Serving.Model Serving ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Ods Dashboard.Model Serving.Model Serving: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] [ WARN ] Exception managing chrome: Permission denied (os error 13) Verify Model Serving Installation :: Verifies that the core compon... | PASS | ------------------------------------------------------------------------------ Verify Openvino_IR Model Via UI :: Test the deployment of an openv... | FAIL | Element 'xpath://h1[text()="Data science projects"]' did not appear in 1 minute 15 seconds. ------------------------------------------------------------------------------ Test Inference Without Token Authentication :: Test the inference ... | FAIL | Page should have contained element 'xpath://div[.="test-model "]' but did not. ------------------------------------------------------------------------------ Model not deployed, skipping deletion step during teardown [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Trdwn: Tests.Ods Dashboard.Model Serving.Model Serving: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Tests.Ods Dashboard.Model Serving.Model Serving | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== Tests.Ods Dashboard.Model Serving.Model Serving Customruntimes :: Collectio... ============================================================================== [ WARN ] Exception managing chrome: Permission denied (os error 13) Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Ods Dashboard.Model Serving.Model Serving Customruntimes: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Verify RHODS Admins Can Import A Custom Serving Runtime Template B... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Verify RHODS Admins Can Delete A Custom Serving Runtime Template [ WARN ] Exception managing chrome: Permission denied (os error 13) [ WARN ] No Modals on the screen right now.. [ WARN ] No Modals on the screen right now.. [ WARN ] No Modals on the screen right now.. | PASS | ------------------------------------------------------------------------------ [ WARN ] msg=There is probably no DS Projects with Diplay Name equal to CustomServingRuntimesProject [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Trdwn: Tests.Ods Dashboard.Model Serving.Model Serving Customruntimes: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Tests.Ods Dashboard.Model Serving.Model Serving Customruntimes :: ... | PASS | 2 tests, 2 passed, 0 failed ============================================================================== Tests.Ods Dashboard.Model Serving | FAIL | 5 tests, 3 passed, 2 failed ============================================================================== Tests.Ods Dashboard.Data Science Pipelines ============================================================================== Tests.Ods Dashboard.Data Science Pipelines.Data-Science-Pipelines-Ui :: Sui... ============================================================================== Installing OpenShift Pipelines operator: pipelines-1.12 (GCP 4.14 Self-Managed: False) Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Error from server (NotFound): deployments.apps "openshift-pipelines-operator" not found Verify User Can Create, Run and Delete A DS Pipeline From DS Proje... | FAIL | Parent suite setup failed: Error installing OpenShift Pipelines operator ------------------------------------------------------------------------------ [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Trdwn: Tests.Ods Dashboard.Data Science Pipelines.Data-Science-Pipelines-Ui: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Tests.Ods Dashboard.Data Science Pipelines.Data-Science-Pipelines-... | FAIL | Suite setup failed: Error installing OpenShift Pipelines operator Also suite teardown failed: Variable '${PROJECTS_TO_DELETE}' not found. 1 test, 0 passed, 1 failed ============================================================================== Tests.Ods Dashboard.Data Science Pipelines | FAIL | 1 test, 0 passed, 1 failed ============================================================================== Tests.Ods Dashboard | FAIL | 19 tests, 16 passed, 3 failed ============================================================================== Tests.Jupyterhub ============================================================================== Tests.Jupyterhub.Base-User-Shutdown-Test :: Test Case that verifies a base ... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Jupyterhub.Base-User-Shutdown-Test: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Verify Base User Can Stop A Running Server :: Verifies that a base... [ WARN ] Exception managing chrome: Permission denied (os error 13) [ ERROR ] Pods not found in search | PASS | ------------------------------------------------------------------------------ Tests.Jupyterhub.Base-User-Shutdown-Test :: Test Case that verifie... | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Jupyterhub.Multiple-Image-Tags :: Test Suite for the multiple noteboo... ============================================================================== Verify All OOTB Images Have Version Dropdowns :: Verifies all imag... Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Jupyterhub.Multiple-Image-Tags: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Jupyterhub.Multiple-Image-Tags :: Test Suite for the multipl... | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Jupyterhub.Special-User-Testing ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Jupyterhub.Special-User-Testing: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Test Special Usernames [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Jupyterhub.Special-User-Testing | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Jupyterhub.Test ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Jupyterhub.Test: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Logged Into OpenShift [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Can Launch Jupyterhub | PASS | ------------------------------------------------------------------------------ Can Login To Jupyterhub | PASS | ------------------------------------------------------------------------------ Can Spawn Notebook [ WARN ] Executing keyword 'Wait Until Page Contains' failed: Text 'Log in with OpenShift' did not appear in 15 seconds. | PASS | ------------------------------------------------------------------------------ Tests.Jupyterhub.Test | PASS | 4 tests, 4 passed, 0 failed ============================================================================== Tests.Jupyterhub | PASS | 7 tests, 7 passed, 0 failed ============================================================================== Tests.Must Gather ============================================================================== Tests.Must Gather.Test-Must-Gather-Logs ============================================================================== Verify that the must-gather image provides RHODS logs and info | PASS | ------------------------------------------------------------------------------ Tests.Must Gather.Test-Must-Gather-Logs | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Must Gather | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps ============================================================================== Tests.Ai Apps.Anaconda Commercial Edition ============================================================================== Tests.Ai Apps.Anaconda Commercial Edition.Anaconda Commercial Edition :: Te... ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Ai Apps.Anaconda Commercial Edition.Anaconda Commercial Edition: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Verify Anaconda Professional Is Available In RHODS Dashboard Explo... [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Ai Apps.Anaconda Commercial Edition.Anaconda Commercial Edit... | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Anaconda Commercial Edition | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Ibm Watson Studio ============================================================================== Tests.Ai Apps.Ibm Watson Studio.Ibm Watson Studio ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Ai Apps.Ibm Watson Studio.Ibm Watson Studio: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Verify IBM Watson Studio Is Available In RHODS Dashboard Explore Page [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Ai Apps.Ibm Watson Studio.Ibm Watson Studio | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Ibm Watson Studio | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Intel Aikit ============================================================================== Tests.Ai Apps.Intel Aikit.Intel Aikit ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Ai Apps.Intel Aikit.Intel Aikit: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Verify Intel AIKIT Is Available In RHODS Dashboard Explore Page [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Ai Apps.Intel Aikit.Intel Aikit | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Intel Aikit | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Rhoam ============================================================================== Tests.Ai Apps.Rhoam.Rhoam ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Ai Apps.Rhoam.Rhoam: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Verify RHOAM Is Available In RHODS Dashboard Explore Page [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Ai Apps.Rhoam.Rhoam | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Rhoam | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Openvino ============================================================================== Tests.Ai Apps.Openvino.Openvino ============================================================================== Prometheus is running in the namespace: redhat-ods-monitoring [ WARN ] Executing keyword 'Alerts Should Not Be Firing' failed: Suite Setup: Tests.Ai Apps.Openvino.Openvino: Alerts should not be firing: ['CodeFlare Operator is not running (for:0, state:firing)', 'CodeFlare Operator taking too long to be up (for:0, state:firing)'] Verify OpenVino Is Available In RHODS Dashboard Explore Page [ WARN ] Exception managing chrome: Permission denied (os error 13) | PASS | ------------------------------------------------------------------------------ Tests.Ai Apps.Openvino.Openvino | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps.Openvino | PASS | 1 test, 1 passed, 0 failed ============================================================================== Tests.Ai Apps | PASS | 5 tests, 5 passed, 0 failed ============================================================================== Tests | FAIL | 41 tests, 35 passed, 6 failed ============================================================================== Output: /logs/artifacts/results/ods-ci-2023-11-08-09-57-TGPLO12Uyp/output.xml XUnit: /logs/artifacts/results/ods-ci-2023-11-08-09-57-TGPLO12Uyp/xunit_test_result.xml Log: /logs/artifacts/results/ods-ci-2023-11-08-09-57-TGPLO12Uyp/log.html Report: /logs/artifacts/results/ods-ci-2023-11-08-09-57-TGPLO12Uyp/test_report.html 6