Description
CRW nightly tests took about 9 hours to finish execution.
We could shorten the test execution time if run the tests in a parallel, taking into account resource limits in QE OCP and on PSI OpenStack.
An example of running the tests in parallel:
- https://main-jenkins-csb-crwqe.apps.ocp4.prod.psi.redhat.com/blue/organizations/jenkins/CodereadyWorkspaces%2Fe2e%2Fcomplex%2Fcrwctl-flow%2Fcrw-crwctl-no-oauth/detail/crw-crwctl-no-oauth/95/pipeline
- https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/tree/CRW-1748
stage("Run E2E tests") { steps { script { def stepsToRun = [:] stepsToRun["Run E2E devfile tests"] = { basicCrwPipelineHelper.runAllDevfileTests( productNamespace: namespace, ocpVersion: ocpVersion, ocpInfra: ocpInfra, productUrl: productUrl, testRepoAddress: testRepoAddress, testRepoBranch: testRepoBranch, testRootDir: testRootDir, e2eTestImage: e2eTestImage, e2eTestParameters: "-e TS_SELENIUM_VALUE_OPENSHIFT_OAUTH=false -e TS_SELENIUM_OCP_USERNAME=$openshiftDto.clusterAdminName -e TS_SELENIUM_OCP_PASSWORD=$openshiftDto.clusterAdminPassword" ) } stepsToRun["Run E2E plugin tests"] = { basicCrwPipelineHelper.runAllPluginTests( productNamespace: namespace, ocpVersion: ocpVersion, ocpInfra: ocpInfra, productUrl: productUrl, testRepoAddress: testRepoAddress, testRepoBranch: testRepoBranch, testRootDir: testRootDir, e2eTestImage: e2eTestImage, e2eTestParameters: "-e TS_SELENIUM_VALUE_OPENSHIFT_OAUTH=false -e TS_TEST_OPENSHIFT_PLUGIN_USERNAME=$openshiftDto.clusterAdminName -e TS_TEST_OPENSHIFT_PLUGIN_PASSWORD=$openshiftDto.clusterAdminPassword " ) } stepsToRun["failFast"] = false parallel stepsToRun } } } }
Next problems of parallel test execution have occurred https://main-jenkins-csb-crwqe.apps.ocp4.prod.psi.redhat.com/job/CodereadyWorkspaces/job/e2e/job/basic/job/typescript-tests/4335/console
1. Tests failed on login step.
2. Test called method "TestWorkspaceUtil.cleanUpAllWorkspaces()" and failed.
Root cause: E2E tests are being run with "-e DELETE_WORKSPACE_ON_FAILED_TEST=true"
✓ Create workspace using API (69ms) Open workspace ▼ DriverHelper.navigateToUrl https://codeready-crw-crwctl-no-oauth.apps.ocp46.crw-qe.com/dashboard/#/ide/admin/xml-plugin-test ✓ Open workspace (181ms) Wait until created workspace is started ▼ Ide.waitAndSwitchToIdeFrame ‣ DriverHelper.waitAndSwitchToFrame By(css selector, iframe#ide-iframe) [ERROR] Switching to IDE frame failed after 900000 timeout. 1) Wait until created workspace is started [ERROR] Test is failed. Additional information (logs, dumps, screenshots) you can find in the "./report/The_"VscodeXmlPlugin"_userstory_Create_workspace_Wait_until_created_workspace_is_started" folder. 3 passing (15m) 1 failing 1) The "VscodeXmlPlugin" userstory Create workspace Wait until created workspace is started: TimeoutError: Waiting to be able to switch to frame Wait timed out after 900010ms at /tmp/e2e-codeready/.tmp/che/tests/e2e/node_modules/selenium-webdriver/lib/promise.js:2201:17 at ManagedPromise.invokeCallback_ (/tmp/e2e-codeready/.tmp/che/tests/e2e/node_modules/selenium-webdriver/lib/promise.js:1376:14) at TaskQueue.execute_ (/tmp/e2e-codeready/.tmp/che/tests/e2e/node_modules/selenium-webdriver/lib/promise.js:3084:14) at TaskQueue.executeNext_ (/tmp/e2e-codeready/.tmp/che/tests/e2e/node_modules/selenium-webdriver/lib/promise.js:3067:27) at /tmp/e2e-codeready/.tmp/che/tests/e2e/node_modules/selenium-webdriver/lib/promise.js:2927:27 at /tmp/e2e-codeready/.tmp/che/tests/e2e/node_modules/selenium-webdriver/lib/promise.js:668:7 at runMicrotasks (<anonymous>) at processTicksAndRejections (internal/process/task_queues.js:93:5) From: Task: Waiting to be able to switch to frame at scheduleWait (/tmp/e2e-codeready/.tmp/che/tests/e2e/node_modules/selenium-webdriver/lib/promise.js:2188:20) at ControlFlow.wait (/tmp/e2e-codeready/.tmp/che/tests/e2e/node_modules/selenium-webdriver/lib/promise.js:2517:12) at thenableWebDriverProxy.wait (/tmp/e2e-codeready/.tmp/che/tests/e2e/node_modules/selenium-webdriver/lib/webdriver.js:934:29) at DriverHelper.waitAndSwitchToFrame (/tmp/e2e-codeready/.tmp/che/tests/e2e/dist/utils/DriverHelper.js:468:27) at Ide.waitAndSwitchToIdeFrame (/tmp/e2e-codeready/.tmp/che/tests/e2e/dist/pageobjects/ide/Ide.js:53:37) at /tmp/e2e-codeready/dist/tests/plugins/VscodeXmlPlugin.spec.js:46:23 at Generator.next (<anonymous>) at /tmp/e2e-codeready/dist/tests/plugins/VscodeXmlPlugin.spec.js:17:71 at new Promise (<anonymous>) at __awaiter (/tmp/e2e-codeready/dist/tests/plugins/VscodeXmlPlugin.spec.js:13:12) at Context.<anonymous> (/tmp/e2e-codeready/dist/tests/plugins/VscodeXmlPlugin.spec.js:45:63) at callFn (/tmp/e2e-codeready/node_modules/mocha/lib/runnable.js:387:21) at Test.Runnable.run (/tmp/e2e-codeready/node_modules/mocha/lib/runnable.js:379:7) at Runner.runTest (/tmp/e2e-codeready/node_modules/mocha/lib/runner.js:535:10) at /tmp/e2e-codeready/node_modules/mocha/lib/runner.js:653:12 at next (/tmp/e2e-codeready/node_modules/mocha/lib/runner.js:447:14) at /tmp/e2e-codeready/node_modules/mocha/lib/runner.js:457:7 at next (/tmp/e2e-codeready/node_modules/mocha/lib/runner.js:362:14) at Immediate.<anonymous> (/tmp/e2e-codeready/node_modules/mocha/lib/runner.js:425:5) at processImmediate (internal/timers.js:461:21) (node:327) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead. ‣ DriverHelper.getDriver Property DELETE_WORKSPACE_ON_FAILED_TEST se to true - trying to stop and delete running workspace. ▼ TestWorkspaceUtil.cleanUpAllWorkspaces ▼ TestWorkspaceUtil.getIdOfRunningWorkspaces [WARN] AccessToken is no longer active, checking refreshToken validity... ▼ Received accessToken from openid-connect using RefreshToken. ▼ Issued token at: 1618946826, access token expires in 300 seconds, refresh token expires in 1800 seconds RequestHandler request: