Using custom assembly from /home/user/codeready Found a custom cert. Adding it to java trust store /home/user/cacerts based on /usr/lib/jvm/jre/lib/security/cacerts Trust this certificate? [no]: Certificate was added to keystore NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED 27-Apr-2021 10:17:10.785 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version name: Apache Tomcat/8.5.61 27-Apr-2021 10:17:10.790 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: Dec 3 2020 14:03:28 UTC 27-Apr-2021 10:17:10.793 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version number: 8.5.61.0 27-Apr-2021 10:17:10.794 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux 27-Apr-2021 10:17:10.794 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 4.18.0-240.15.1.el8_3.x86_64 27-Apr-2021 10:17:10.794 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64 27-Apr-2021 10:17:10.794 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /usr/lib/jvm/java-11-openjdk-11.0.11.0.9-0.el8_3.x86_64 27-Apr-2021 10:17:10.794 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 11.0.11+9-LTS 27-Apr-2021 10:17:10.795 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Red Hat, Inc. 27-Apr-2021 10:17:10.795 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: /home/user/codeready/tomcat 27-Apr-2021 10:17:10.795 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: /home/user/codeready/tomcat 27-Apr-2021 10:17:10.796 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED 27-Apr-2021 10:17:10.796 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io=ALL-UNNAMED 27-Apr-2021 10:17:10.797 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED 27-Apr-2021 10:17:10.797 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/home/user/codeready/tomcat/conf/logging.properties 27-Apr-2021 10:17:10.797 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager 27-Apr-2021 10:17:10.797 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:MaxRAMPercentage=85.0 27-Apr-2021 10:17:10.797 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dche.docker.network=bridge 27-Apr-2021 10:17:10.798 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djavax.net.ssl.trustStore=/home/user/cacerts 27-Apr-2021 10:17:10.798 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djavax.net.ssl.trustStorePassword=changeit 27-Apr-2021 10:17:10.798 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dport.http=8080 27-Apr-2021 10:17:10.798 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dche.home=/home/user/codeready 27-Apr-2021 10:17:10.799 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dche.logs.dir=/logs/ 27-Apr-2021 10:17:10.799 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dche.logs.level=INFO 27-Apr-2021 10:17:10.799 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djuli-logback.configurationFile=file:/home/user/codeready/tomcat/conf/tomcat-logger.xml 27-Apr-2021 10:17:10.799 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048 27-Apr-2021 10:17:10.799 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources 27-Apr-2021 10:17:10.800 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0022 27-Apr-2021 10:17:10.800 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dche.local.conf.dir=/home/user/codeready/tomcat/conf/ 27-Apr-2021 10:17:10.800 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs= 27-Apr-2021 10:17:10.800 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/home/user/codeready/tomcat 27-Apr-2021 10:17:10.801 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/home/user/codeready/tomcat 27-Apr-2021 10:17:10.848 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/home/user/codeready/tomcat/temp 27-Apr-2021 10:17:10.899 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"] 27-Apr-2021 10:17:10.964 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read 27-Apr-2021 10:17:11.053 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 1487 ms 27-Apr-2021 10:17:11.172 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina] 27-Apr-2021 10:17:11.172 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.5.61 27-Apr-2021 10:17:11.283 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/home/user/codeready/tomcat/webapps/ROOT.war] 27-Apr-2021 10:17:16.674 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/home/user/codeready/tomcat/webapps/ROOT.war] has finished in [5,391] ms 27-Apr-2021 10:17:16.676 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/home/user/codeready/tomcat/webapps/api.war] 2021-04-27 10:17:25,778[ost-startStop-1] [INFO ] [.e.c.c.d.JNDIDataSourceFactory 63] - This=org.eclipse.che.core.db.postgresql.PostgreSQLJndiDataSourceFactory@6898a1bb obj=ResourceRef[className=javax.sql.DataSource,factoryClassLocation=null,factoryClassName=org.apache.naming.factory.ResourceFactory,{type=scope,content=Shareable},{type=auth,content=Container},{type=singleton,content=true},{type=factory,content=org.eclipse.che.api.CommonJndiDataSourceFactory}] name=che Context=org.apache.naming.NamingContext@1f2192dd environment={} WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.google.inject.assistedinject.FactoryProvider2$MethodHandleWrapper (file:/home/user/codeready/tomcat/webapps/api/WEB-INF/lib/guice-assistedinject-4.2.2.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int) WARNING: Please consider reporting this to the maintainers of com.google.inject.assistedinject.FactoryProvider2$MethodHandleWrapper WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release 2021-04-27 10:17:31,807[ost-startStop-1] [INFO ] [o.e.c.m.k.s.OIDCInfoProvider 72] - Retrieving OpenId configuration from endpoint: http://keycloak.openshift-workspaces.svc:8080/auth/realms/codeready/.well-known/openid-configuration 2021-04-27 10:17:32,659[ost-startStop-1] [INFO ] [o.e.c.m.k.s.OIDCInfoProvider 81] - openid configuration = {issuer=https://keycloak-openshift-workspaces.apps.ocp47-aws.crw-qe.com/auth/realms/codeready, authorization_endpoint=https://keycloak-openshift-workspaces.apps.ocp47-aws.crw-qe.com/auth/realms/codeready/protocol/openid-connect/auth, token_endpoint=http://keycloak.openshift-workspaces.svc:8080/auth/realms/codeready/protocol/openid-connect/token, token_introspection_endpoint=http://keycloak.openshift-workspaces.svc:8080/auth/realms/codeready/protocol/openid-connect/token/introspect, userinfo_endpoint=http://keycloak.openshift-workspaces.svc:8080/auth/realms/codeready/protocol/openid-connect/userinfo, end_session_endpoint=https://keycloak-openshift-workspaces.apps.ocp47-aws.crw-qe.com/auth/realms/codeready/protocol/openid-connect/logout, jwks_uri=http://keycloak.openshift-workspaces.svc:8080/auth/realms/codeready/protocol/openid-connect/certs, check_session_iframe=https://keycloak-openshift-workspaces.apps.ocp47-aws.crw-qe.com/auth/realms/codeready/protocol/openid-connect/login-status-iframe.html, grant_types_supported=[authorization_code, implicit, refresh_token, password, client_credentials], response_types_supported=[code, none, id_token, token, id_token token, code id_token, code token, code id_token token], subject_types_supported=[public, pairwise], id_token_signing_alg_values_supported=[PS384, ES384, RS384, HS256, HS512, ES256, RS256, HS384, ES512, PS256, PS512, RS512], id_token_encryption_alg_values_supported=[RSA-OAEP, RSA1_5], id_token_encryption_enc_values_supported=[A128GCM, A128CBC-HS256], userinfo_signing_alg_values_supported=[PS384, ES384, RS384, HS256, HS512, ES256, RS256, HS384, ES512, PS256, PS512, RS512, none], request_object_signing_alg_values_supported=[PS384, ES384, RS384, HS256, HS512, ES256, RS256, HS384, ES512, PS256, PS512, RS512, none], response_modes_supported=[query, fragment, form_post], registration_endpoint=http://keycloak.openshift-workspaces.svc:8080/auth/realms/codeready/clients-registrations/openid-connect, token_endpoint_auth_methods_supported=[private_key_jwt, client_secret_basic, client_secret_post, tls_client_auth, client_secret_jwt], token_endpoint_auth_signing_alg_values_supported=[PS384, ES384, RS384, HS256, HS512, ES256, RS256, HS384, ES512, PS256, PS512, RS512], claims_supported=[aud, sub, iss, auth_time, name, given_name, family_name, preferred_username, email, acr], claim_types_supported=[normal], claims_parameter_supported=false, scopes_supported=[openid, microprofile-jwt, web-origins, roles, phone, address, email, profile, offline_access], request_parameter_supported=true, request_uri_parameter_supported=true, require_request_uri_registration=true, code_challenge_methods_supported=[plain, S256], tls_client_certificate_bound_access_tokens=true, introspection_endpoint=http://keycloak.openshift-workspaces.svc:8080/auth/realms/codeready/protocol/openid-connect/token/introspect} 2021-04-27 10:17:35,149[ost-startStop-1] [INFO ] [o.j.p.kubernetes.KUBE_PING 131] - namespace openshift-workspaces set; clustering enabled ------------------------------------------------------------------- GMS: address=codeready-795777c57c-2pqrg-10926, cluster=RemoteSubscriptionChannel, physical address=10.129.5.88:7800 ------------------------------------------------------------------- 2021-04-27 10:17:38,762[ost-startStop-1] [INFO ] [o.jgroups.protocols.pbcast.GMS 125] - codeready-795777c57c-2pqrg-10926: no members discovered after 3588 ms: creating cluster as coordinator 2021-04-27 10:17:38,802[ost-startStop-1] [INFO ] [o.j.p.kubernetes.KUBE_PING 131] - namespace openshift-workspaces set; clustering enabled ------------------------------------------------------------------- GMS: address=codeready-795777c57c-2pqrg-22968, cluster=WorkspaceLocks, physical address=10.129.5.88:7801 ------------------------------------------------------------------- 2021-04-27 10:17:41,894[ost-startStop-1] [INFO ] [o.jgroups.protocols.pbcast.GMS 125] - codeready-795777c57c-2pqrg-22968: no members discovered after 3082 ms: creating cluster as coordinator 2021-04-27 10:17:41,905[ost-startStop-1] [INFO ] [o.j.p.kubernetes.KUBE_PING 131] - namespace openshift-workspaces set; clustering enabled ------------------------------------------------------------------- GMS: address=codeready-795777c57c-2pqrg-27395, cluster=WorkspaceStateCache, physical address=10.129.5.88:7802 ------------------------------------------------------------------- 2021-04-27 10:17:44,964[ost-startStop-1] [INFO ] [o.jgroups.protocols.pbcast.GMS 125] - codeready-795777c57c-2pqrg-27395: no members discovered after 3049 ms: creating cluster as coordinator 2021-04-27 10:17:45,221[ost-startStop-1] [INFO ] [o.f.c.i.d.DbSupportFactory 44] - Database: jdbc:postgresql://postgres:5432/dbche (PostgreSQL 9.6) 2021-04-27 10:17:45,287[ost-startStop-1] [INFO ] [o.f.c.i.util.VersionPrinter 44] - Flyway 4.2.0 by Boxfuse 2021-04-27 10:17:45,296[ost-startStop-1] [INFO ] [o.f.c.i.d.DbSupportFactory 44] - Database: jdbc:postgresql://postgres:5432/dbche (PostgreSQL 9.6) 2021-04-27 10:17:45,379[ost-startStop-1] [INFO ] [i.f.CustomSqlMigrationResolver 158] - Searching for SQL scripts in locations [classpath:che-schema] 2021-04-27 10:17:45,561[ost-startStop-1] [INFO ] [o.f.c.i.command.DbValidate 44] - Successfully validated 63 migrations (execution time 00:00.187s) 2021-04-27 10:17:45,585[ost-startStop-1] [INFO ] [o.f.c.i.m.MetaDataTableImpl 44] - Creating Metadata table: "public"."schema_version" 2021-04-27 10:17:45,685[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Current version of schema "public": << Empty Schema >> 2021-04-27 10:17:45,879[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.0.0.8.1 - 1__init.sql 2021-04-27 10:17:46,148[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.0.0.9.1 - 1__add_index_on_workspace_temporary.sql 2021-04-27 10:17:46,178[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.0.0.9.2 - 2__update_local_links_in_environments.sql 2021-04-27 10:17:46,203[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.2.0.1 - 1__increase_project_attributes_values_length.sql 2021-04-27 10:17:46,259[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.4.0.1 - 1__drop_user_to_account_relation.sql 2021-04-27 10:17:46,298[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.4.0.2 - 2__create_missed_account_indexes.sql 2021-04-27 10:17:46,351[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.6.0.1 - 1__add_exec_agent_where_terminal_agent_is_present.sql 2021-04-27 10:17:46,384[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.7.0.1 - 1__add_factory.sql 2021-04-27 10:17:46,506[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.7.0.2 - 2__remove_match_policy.sql 2021-04-27 10:17:46,529[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.8.0.1 - 1__add_foreigh_key_indexes.sql 2021-04-27 10:17:46,660[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.11.0.1 - 1__optimize_user_search.sql 2021-04-27 10:17:46,700[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.19.0.0.1 - 0.1__add_permissions.sql 2021-04-27 10:17:46,883[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.19.0.0.2 - 0.2__add_resources.sql 2021-04-27 10:17:46,921[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 5.19.0.0.3 - 0.3__add_organization.sql 2021-04-27 10:17:46,988[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.1 - 1__add_path_to_serverconf.sql 2021-04-27 10:17:47,004[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.2 - 2__rename_agents_to_installers.sql 2021-04-27 10:17:47,030[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.3 - 3__add_installer.sql 2021-04-27 10:17:47,085[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.3.1 - 3.1__remove_old_recipe_permissions.sql 2021-04-27 10:17:47,108[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.4 - 4__remove_old_recipe.sql 2021-04-27 10:17:47,126[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.5 - 5__add_machine_env.sql 2021-04-27 10:17:47,153[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.6 - 6__remove_snapshots.sql 2021-04-27 10:17:47,173[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.7 - 7__add_machine_volumes.sql 2021-04-27 10:17:47,201[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.8 - 8__add_serverconf_attributes.sql 2021-04-27 10:17:47,235[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.9 - 9__increase_externalmachine_env_value_length.sql 2021-04-27 10:17:47,251[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.10 - 10__move_dockerimage_recipe_location_to_content.sql 2021-04-27 10:17:47,267[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.11 - 11__increase_workspace_attributes_values_length.sql 2021-04-27 10:17:47,282[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.0.0.12 - 12__remove_stack_sources.sql 2021-04-27 10:17:47,297[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.3.0.1 - 1__add_fk_indexes.sql 2021-04-27 10:17:47,313[ost-startStop-1] [WARN ] [o.f.c.i.dbsupport.JdbcTemplate 48] - DB: identifier "che_index_factory_on_projects_loaded_action_value_action_entity_id" will be truncated to "che_index_factory_on_projects_loaded_action_value_action_entity" (SQL State: 42622 - Error Code: 0) 2021-04-27 10:17:47,324[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.3.0.1.1 - 1.1__add_fk_indexes.sql 2021-04-27 10:17:47,361[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.4.0.1 - 1__add_workspace_expirations.sql 2021-04-27 10:17:47,392[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.4.0.2 - 2__add_signature_key.sql 2021-04-27 10:17:47,449[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.4.0.3 - 3__add_k8s_runtimes.sql 2021-04-27 10:17:47,511[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.10.0.1 - 1__add_workspace_cfg_attributes.sql 2021-04-27 10:17:47,536[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.10.0.2 - 2__change_signature_key_pair_id.sql 2021-04-27 10:17:47,566[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.11.0.1 - 1__add_signature_key_constraints.sql 2021-04-27 10:17:47,599[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.12.0.1 - 1__rename_project_attributes_values_field.sql 2021-04-27 10:17:47,615[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.15.0.1 - 1__remove_not_null_constraint_from_env_name_fields.sql 2021-04-27 10:17:47,638[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.15.0.2 - 2__add_commands_to_k8s_runtime.sql 2021-04-27 10:17:47,675[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.16.0.1 - 1__increase_workspace_config_attributes_values_length.sql 2021-04-27 10:17:47,691[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.16.0.2 - 2__create_workspace_activity_table.sql 2021-04-27 10:17:47,738[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.16.0.3 - 3__bootstrap_ws_activity_data.sql 2021-04-27 10:17:47,755[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 6.17.0.1 - 1__convert_enums_to_strings.sql 2021-04-27 10:17:47,782[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.0.0.4.0.1 - 1__add_devfile.sql 2021-04-27 10:17:47,970[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.0.0.5.0.1 - 1__devfile_command_reference.sql 2021-04-27 10:17:47,994[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.0.0.6.0.1 - 1__add_devfile_component_prefs.sql 2021-04-27 10:17:48,019[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.0.0.7.0.1 - 1__add_registry_url_to_devfile_component.sql 2021-04-27 10:17:48,049[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.0.0.8.0.2.0.1 - 1__devfile_metadata.sql 2021-04-27 10:17:48,072[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.0.0.8.0.2.0.2 - 2__devfile_make_some_fields_optional.sql 2021-04-27 10:17:48,088[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.1.0.1 - 1__change_devfile_component_preferences_type.sql 2021-04-27 10:17:48,105[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.1.0.1.1 - 1.1__remove_stack_permissions.sql 2021-04-27 10:17:48,123[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.1.0.2 - 2__remove_stacks.sql 2021-04-27 10:17:48,142[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.2.0.1 - 1__remove_installers.sql 2021-04-27 10:17:48,167[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.4.0.1 - 1__add_devfile_source_sparse_checkout_dir.sql 2021-04-27 10:17:48,183[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.4.0.2 - 2__add_preview_url_to_devfile_command.sql 2021-04-27 10:17:48,205[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.6.0.1 - 1__drop_che_workspace_expiration.sql 2021-04-27 10:17:48,221[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.10.0.1 - 1__add_devfile_plugin_editor_component_cpu_limit_request.sql 2021-04-27 10:17:48,236[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.10.0.2 - 2__add_devfile_plugin_editor_component_ram_request.sql 2021-04-27 10:17:48,251[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.11.0.1 - 1__update_inconsistent_stopped_workspace_activities.sql 2021-04-27 10:17:48,267[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.16.0.1 - 1__add_devfile_component_automount_workspace_secrets.sql 2021-04-27 10:17:48,293[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.20.0.1 - 1__userdevfile.sql 2021-04-27 10:17:48,332[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.20.0.1.1 - 1.1__add_userdevfile_permissions.sql 2021-04-27 10:17:48,378[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.21.0.1 - 1__remove_installers.sql 2021-04-27 10:17:48,397[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Migrating schema "public" to version 7.26.0.1 - 1__remove_factory_button_and_image.sql 2021-04-27 10:17:48,424[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Successfully applied 63 migrations to schema "public" (execution time 00:02.841s). 2021-04-27 10:17:50,490[ost-startStop-1] [INFO ] [o.j.p.kubernetes.KUBE_PING 131] - namespace openshift-workspaces set; clustering enabled ------------------------------------------------------------------- GMS: address=codeready-795777c57c-2pqrg-58415, cluster=EclipseLinkCommandChannel, physical address=10.129.5.88:7803 ------------------------------------------------------------------- 2021-04-27 10:17:53,589[ost-startStop-1] [INFO ] [o.jgroups.protocols.pbcast.GMS 125] - codeready-795777c57c-2pqrg-58415: no members discovered after 3091 ms: creating cluster as coordinator 2021-04-27 10:17:53,655[ost-startStop-1] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 175] - Configured factories for environments: '[openshift, kubernetes, no-environment]' 2021-04-27 10:17:53,656[ost-startStop-1] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 176] - Registered infrastructure 'openshift' 2021-04-27 10:17:53,869[ost-startStop-1] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 702] - Infrastructure is tracking 0 active runtimes 2021-04-27 10:17:53,953[ost-startStop-1] [INFO ] [o.e.c.a.c.u.ApiInfoLogInformer 36] - Eclipse Che Api Core: Build info '7.28.3-SNAPSHOT' scmRevision 'a6b3b05bfd455bd83b0d66aad23ff8a106b59f56' implementationVersion '7.28.3-SNAPSHOT' 2021-04-27 10:17:53,980[ost-startStop-1] [WARN ] [p.s.AdminPermissionInitializer 69] - Admin admin not found yet. 2021-04-27 10:17:54,005[ost-startStop-1] [INFO ] [o.e.c.c.metrics.MetricsServer 46] - Metrics server started at port 8087 successfully 27-Apr-2021 10:17:54.567 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/home/user/codeready/tomcat/webapps/api.war] has finished in [37,890] ms 27-Apr-2021 10:17:54.567 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/home/user/codeready/tomcat/webapps/swagger.war] 27-Apr-2021 10:17:54.681 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/home/user/codeready/tomcat/webapps/swagger.war] has finished in [114] ms 27-Apr-2021 10:17:54.683 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/home/user/codeready/tomcat/webapps/dashboard] 27-Apr-2021 10:17:54.756 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time. 27-Apr-2021 10:17:54.759 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/home/user/codeready/tomcat/webapps/dashboard] has finished in [76] ms 27-Apr-2021 10:17:54.765 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"] 27-Apr-2021 10:17:54.776 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 43722 ms 2021-04-27 10:23:00,340[nio-8080-exec-4] [INFO ] [o.e.c.a.w.s.WorkspaceManager 694] - Workspace 'admin/java-lombok-rrtqa' with id 'workspace5frdg50h50khqq6w' created by user 'admin' 2021-04-27 10:23:03,735[io-8080-exec-10] [WARN ] [.c.w.i.k.n.KubernetesNamespace 193] - Can't label the namespace due to lack of permissions. Grant cluster-wide permissions to `get` and `update` the `namespaces` to the `che` service account (Che operator might have already prepared a cluster role called `che-namespace-editor` for this, depending on its configuration). Alternatively, consider disabling the feature by setting `che.infra.kubernetes.namepsace.label` to `false`. 2021-04-27 10:23:03,979[io-8080-exec-10] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 480] - Starting workspace 'admin/java-lombok-rrtqa' with id 'workspace5frdg50h50khqq6w' by user 'admin' 2021-04-27 10:23:04,779[aceSharedPool-0] [WARN ] [.c.w.i.k.n.KubernetesNamespace 193] - Can't label the namespace due to lack of permissions. Grant cluster-wide permissions to `get` and `update` the `namespaces` to the `che` service account (Che operator might have already prepared a cluster role called `che-namespace-editor` for this, depending on its configuration). Alternatively, consider disabling the feature by setting `che.infra.kubernetes.namepsace.label` to `false`. 2021-04-27 10:23:05,948[aceSharedPool-0] [WARN ] [.c.w.i.k.n.KubernetesNamespace 193] - Can't label the namespace due to lack of permissions. Grant cluster-wide permissions to `get` and `update` the `namespaces` to the `che` service account (Che operator might have already prepared a cluster role called `che-namespace-editor` for this, depending on its configuration). Alternatively, consider disabling the feature by setting `che.infra.kubernetes.namepsace.label` to `false`. 2021-04-27 10:28:06,385[aceSharedPool-0] [ERROR] [e.c.w.i.k.n.p.PVCSubPathHelper 223] - Unable to perform '[mkdir, -m, 777, -p, /tmp/job_mount/workspace5frdg50h50khqq6w/che-logs-che-plugin-broker/]' command for the workspace 'workspace5frdg50h50khqq6w' cause: 'Waiting for pod 'mkdir-workspace5frdg50h50khqq6w' reached timeout' 2021-04-27 10:28:06,406[aceSharedPool-0] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:841) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.PVCSubPathHelper.execute(PVCSubPathHelper.java:233) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.PVCSubPathHelper.execute(PVCSubPathHelper.java:167) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.PVCSubPathHelper.createDirs(PVCSubPathHelper.java:136) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.CommonPVCStrategy.prepare(CommonPVCStrategy.java:221) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:67) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,412[aceSharedPool-0] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.create(KubernetesConfigsMaps.java:70) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:98) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,417[aceSharedPool-0] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:875) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:140) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,418[aceSharedPool-0] [ERROR] [o.e.c.w.i.k.w.b.DeployBroker 142] - Brokers pod removal failed. Error: Your session has expired. Please login to Che again to get access to your OpenShift account org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:875) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:140) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,422[aceSharedPool-0] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesSecrets.delete(KubernetesSecrets.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:145) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,423[aceSharedPool-0] [ERROR] [o.e.c.w.i.k.w.b.DeployBroker 147] - Brokers secret removal failed. Error: Your session has expired. Please login to Che again to get access to your OpenShift account org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesSecrets.delete(KubernetesSecrets.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:145) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,427[aceSharedPool-0] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.delete(KubernetesConfigsMaps.java:87) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:150) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,428[aceSharedPool-0] [ERROR] [o.e.c.w.i.k.w.b.DeployBroker 152] - Brokers config map removal failed. Error: Your session has expired. Please login to Che again to get access to your OpenShift account org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.delete(KubernetesConfigsMaps.java:87) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:150) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,428[aceSharedPool-0] [WARN ] [.i.k.KubernetesInternalRuntime 257] - Failed to start Kubernetes runtime of workspace workspace5frdg50h50khqq6w. org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.create(KubernetesConfigsMaps.java:70) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:98) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,432[aceSharedPool-0] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.createOC(OpenShiftClientFactory.java:97) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftRoutes.delete(OpenShiftRoutes.java:84) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,437[aceSharedPool-0] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesServices.delete(KubernetesServices.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,442[aceSharedPool-0] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:875) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,446[aceSharedPool-0] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesSecrets.delete(KubernetesSecrets.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,449[aceSharedPool-0] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.delete(KubernetesConfigsMaps.java:87) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,450[aceSharedPool-0] [WARN ] [.i.k.KubernetesInternalRuntime 268] - Failed to clean up namespace after workspace 'workspace5frdg50h50khqq6w' start failing. org.eclipse.che.api.workspace.server.spi.InfrastructureException: Error(s) occurs while cleaning up the namespace. Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:302) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:28:06,470[aceSharedPool-0] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 995] - Workspace 'admin:java-lombok-rrtqa' with id 'workspace5frdg50h50khqq6w' start failed 2021-04-27 10:36:54,221[nio-8080-exec-9] [WARN ] [.c.w.i.k.n.KubernetesNamespace 193] - Can't label the namespace due to lack of permissions. Grant cluster-wide permissions to `get` and `update` the `namespaces` to the `che` service account (Che operator might have already prepared a cluster role called `che-namespace-editor` for this, depending on its configuration). Alternatively, consider disabling the feature by setting `che.infra.kubernetes.namepsace.label` to `false`. 2021-04-27 10:36:54,374[nio-8080-exec-9] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 480] - Starting workspace 'admin/java-lombok-rrtqa' with id 'workspace5frdg50h50khqq6w' by user 'admin' 2021-04-27 10:36:59,648[aceSharedPool-1] [WARN ] [.c.w.i.k.n.KubernetesNamespace 193] - Can't label the namespace due to lack of permissions. Grant cluster-wide permissions to `get` and `update` the `namespaces` to the `che` service account (Che operator might have already prepared a cluster role called `che-namespace-editor` for this, depending on its configuration). Alternatively, consider disabling the feature by setting `che.infra.kubernetes.namepsace.label` to `false`. 2021-04-27 10:36:59,809[aceSharedPool-1] [WARN ] [.c.w.i.k.n.KubernetesNamespace 193] - Can't label the namespace due to lack of permissions. Grant cluster-wide permissions to `get` and `update` the `namespaces` to the `che` service account (Che operator might have already prepared a cluster role called `che-namespace-editor` for this, depending on its configuration). Alternatively, consider disabling the feature by setting `che.infra.kubernetes.namepsace.label` to `false`. 2021-04-27 10:42:00,011[aceSharedPool-1] [ERROR] [e.c.w.i.k.n.p.PVCSubPathHelper 223] - Unable to perform '[mkdir, -m, 777, -p, /tmp/job_mount/workspace5frdg50h50khqq6w/che-logs-che-plugin-broker/]' command for the workspace 'workspace5frdg50h50khqq6w' cause: 'Waiting for pod 'mkdir-workspace5frdg50h50khqq6w' reached timeout' 2021-04-27 10:42:00,025[aceSharedPool-1] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:841) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.PVCSubPathHelper.execute(PVCSubPathHelper.java:233) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.PVCSubPathHelper.execute(PVCSubPathHelper.java:167) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.PVCSubPathHelper.createDirs(PVCSubPathHelper.java:136) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.CommonPVCStrategy.prepare(CommonPVCStrategy.java:221) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:67) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,031[aceSharedPool-1] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.create(KubernetesConfigsMaps.java:70) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:98) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,037[aceSharedPool-1] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:875) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:140) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,037[aceSharedPool-1] [ERROR] [o.e.c.w.i.k.w.b.DeployBroker 142] - Brokers pod removal failed. Error: Your session has expired. Please login to Che again to get access to your OpenShift account org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:875) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:140) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,042[aceSharedPool-1] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesSecrets.delete(KubernetesSecrets.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:145) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,042[aceSharedPool-1] [ERROR] [o.e.c.w.i.k.w.b.DeployBroker 147] - Brokers secret removal failed. Error: Your session has expired. Please login to Che again to get access to your OpenShift account org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesSecrets.delete(KubernetesSecrets.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:145) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,047[aceSharedPool-1] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.delete(KubernetesConfigsMaps.java:87) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:150) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,047[aceSharedPool-1] [ERROR] [o.e.c.w.i.k.w.b.DeployBroker 152] - Brokers config map removal failed. Error: Your session has expired. Please login to Che again to get access to your OpenShift account org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.delete(KubernetesConfigsMaps.java:87) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:150) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,047[aceSharedPool-1] [WARN ] [.i.k.KubernetesInternalRuntime 257] - Failed to start Kubernetes runtime of workspace workspace5frdg50h50khqq6w. org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.create(KubernetesConfigsMaps.java:70) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:98) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,052[aceSharedPool-1] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.createOC(OpenShiftClientFactory.java:97) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftRoutes.delete(OpenShiftRoutes.java:84) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,056[aceSharedPool-1] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesServices.delete(KubernetesServices.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,063[aceSharedPool-1] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:875) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,069[aceSharedPool-1] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesSecrets.delete(KubernetesSecrets.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,079[aceSharedPool-1] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.delete(KubernetesConfigsMaps.java:87) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,079[aceSharedPool-1] [WARN ] [.i.k.KubernetesInternalRuntime 268] - Failed to clean up namespace after workspace 'workspace5frdg50h50khqq6w' start failing. org.eclipse.che.api.workspace.server.spi.InfrastructureException: Error(s) occurs while cleaning up the namespace. Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:302) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 10:42:00,093[aceSharedPool-1] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 995] - Workspace 'admin:java-lombok-rrtqa' with id 'workspace5frdg50h50khqq6w' start failed 2021-04-27 11:01:18,753[nio-8080-exec-8] [WARN ] [.c.w.i.k.n.KubernetesNamespace 193] - Can't label the namespace due to lack of permissions. Grant cluster-wide permissions to `get` and `update` the `namespaces` to the `che` service account (Che operator might have already prepared a cluster role called `che-namespace-editor` for this, depending on its configuration). Alternatively, consider disabling the feature by setting `che.infra.kubernetes.namepsace.label` to `false`. 2021-04-27 11:01:18,918[nio-8080-exec-8] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 480] - Starting workspace 'admin/java-lombok-rrtqa' with id 'workspace5frdg50h50khqq6w' by user 'admin' 2021-04-27 11:01:29,671[aceSharedPool-2] [WARN ] [.c.w.i.k.n.KubernetesNamespace 193] - Can't label the namespace due to lack of permissions. Grant cluster-wide permissions to `get` and `update` the `namespaces` to the `che` service account (Che operator might have already prepared a cluster role called `che-namespace-editor` for this, depending on its configuration). Alternatively, consider disabling the feature by setting `che.infra.kubernetes.namepsace.label` to `false`. 2021-04-27 11:01:29,827[aceSharedPool-2] [WARN ] [.c.w.i.k.n.KubernetesNamespace 193] - Can't label the namespace due to lack of permissions. Grant cluster-wide permissions to `get` and `update` the `namespaces` to the `che` service account (Che operator might have already prepared a cluster role called `che-namespace-editor` for this, depending on its configuration). Alternatively, consider disabling the feature by setting `che.infra.kubernetes.namepsace.label` to `false`. 2021-04-27 11:06:30,176[aceSharedPool-2] [ERROR] [e.c.w.i.k.n.p.PVCSubPathHelper 223] - Unable to perform '[mkdir, -m, 777, -p, /tmp/job_mount/workspace5frdg50h50khqq6w/che-logs-che-plugin-broker/]' command for the workspace 'workspace5frdg50h50khqq6w' cause: 'Waiting for pod 'mkdir-workspace5frdg50h50khqq6w' reached timeout' 2021-04-27 11:06:32,694[aceSharedPool-2] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:841) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.PVCSubPathHelper.execute(PVCSubPathHelper.java:233) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.PVCSubPathHelper.execute(PVCSubPathHelper.java:167) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.PVCSubPathHelper.createDirs(PVCSubPathHelper.java:136) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.CommonPVCStrategy.prepare(CommonPVCStrategy.java:221) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:67) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,697[aceSharedPool-2] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.create(KubernetesConfigsMaps.java:70) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:98) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,700[aceSharedPool-2] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:875) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:140) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,701[aceSharedPool-2] [ERROR] [o.e.c.w.i.k.w.b.DeployBroker 142] - Brokers pod removal failed. Error: Your session has expired. Please login to Che again to get access to your OpenShift account org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:875) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:140) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,704[aceSharedPool-2] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesSecrets.delete(KubernetesSecrets.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:145) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,704[aceSharedPool-2] [ERROR] [o.e.c.w.i.k.w.b.DeployBroker 147] - Brokers secret removal failed. Error: Your session has expired. Please login to Che again to get access to your OpenShift account org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesSecrets.delete(KubernetesSecrets.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:145) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,707[aceSharedPool-2] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.delete(KubernetesConfigsMaps.java:87) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:150) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,707[aceSharedPool-2] [ERROR] [o.e.c.w.i.k.w.b.DeployBroker 152] - Brokers config map removal failed. Error: Your session has expired. Please login to Che again to get access to your OpenShift account org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.delete(KubernetesConfigsMaps.java:87) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:150) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,707[aceSharedPool-2] [WARN ] [.i.k.KubernetesInternalRuntime 257] - Failed to start Kubernetes runtime of workspace workspace5frdg50h50khqq6w. org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.create(KubernetesConfigsMaps.java:70) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:98) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,711[aceSharedPool-2] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.createOC(OpenShiftClientFactory.java:97) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftRoutes.delete(OpenShiftRoutes.java:84) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,715[aceSharedPool-2] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesServices.delete(KubernetesServices.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,720[aceSharedPool-2] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:875) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,724[aceSharedPool-2] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesSecrets.delete(KubernetesSecrets.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,728[aceSharedPool-2] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.delete(KubernetesConfigsMaps.java:87) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,728[aceSharedPool-2] [WARN ] [.i.k.KubernetesInternalRuntime 268] - Failed to clean up namespace after workspace 'workspace5frdg50h50khqq6w' start failing. org.eclipse.che.api.workspace.server.spi.InfrastructureException: Error(s) occurs while cleaning up the namespace. Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:302) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:06:32,739[aceSharedPool-2] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 995] - Workspace 'admin:java-lombok-rrtqa' with id 'workspace5frdg50h50khqq6w' start failed 2021-04-27 11:07:31,298[nio-8080-exec-9] [INFO ] [o.e.c.a.w.s.WorkspaceManager 694] - Workspace 'admin/vertx-health-checks-4cyx2' with id 'workspace0kv0zlnibbphjd8l' created by user 'admin' 2021-04-27 11:07:32,396[io-8080-exec-10] [WARN ] [.c.w.i.k.n.KubernetesNamespace 193] - Can't label the namespace due to lack of permissions. Grant cluster-wide permissions to `get` and `update` the `namespaces` to the `che` service account (Che operator might have already prepared a cluster role called `che-namespace-editor` for this, depending on its configuration). Alternatively, consider disabling the feature by setting `che.infra.kubernetes.namepsace.label` to `false`. 2021-04-27 11:07:32,554[io-8080-exec-10] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 480] - Starting workspace 'admin/vertx-health-checks-4cyx2' with id 'workspace0kv0zlnibbphjd8l' by user 'admin' 2021-04-27 11:07:32,738[aceSharedPool-3] [WARN ] [.c.w.i.k.n.KubernetesNamespace 193] - Can't label the namespace due to lack of permissions. Grant cluster-wide permissions to `get` and `update` the `namespaces` to the `che` service account (Che operator might have already prepared a cluster role called `che-namespace-editor` for this, depending on its configuration). Alternatively, consider disabling the feature by setting `che.infra.kubernetes.namepsace.label` to `false`. 2021-04-27 11:07:32,896[aceSharedPool-3] [WARN ] [.c.w.i.k.n.KubernetesNamespace 193] - Can't label the namespace due to lack of permissions. Grant cluster-wide permissions to `get` and `update` the `namespaces` to the `che` service account (Che operator might have already prepared a cluster role called `che-namespace-editor` for this, depending on its configuration). Alternatively, consider disabling the feature by setting `che.infra.kubernetes.namepsace.label` to `false`. 2021-04-27 11:12:33,168[aceSharedPool-3] [ERROR] [e.c.w.i.k.n.p.PVCSubPathHelper 223] - Unable to perform '[mkdir, -m, 777, -p, /tmp/job_mount/workspace0kv0zlnibbphjd8l/che-logs-che-plugin-broker/]' command for the workspace 'workspace0kv0zlnibbphjd8l' cause: 'Waiting for pod 'mkdir-workspace0kv0zlnibbphjd8l' reached timeout' 2021-04-27 11:12:33,178[aceSharedPool-3] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:841) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.PVCSubPathHelper.execute(PVCSubPathHelper.java:233) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.PVCSubPathHelper.execute(PVCSubPathHelper.java:167) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.PVCSubPathHelper.createDirs(PVCSubPathHelper.java:136) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.pvc.CommonPVCStrategy.prepare(CommonPVCStrategy.java:221) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:67) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,182[aceSharedPool-3] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.create(KubernetesConfigsMaps.java:70) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:98) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,186[aceSharedPool-3] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:875) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:140) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,186[aceSharedPool-3] [ERROR] [o.e.c.w.i.k.w.b.DeployBroker 142] - Brokers pod removal failed. Error: Your session has expired. Please login to Che again to get access to your OpenShift account org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:875) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:140) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,190[aceSharedPool-3] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesSecrets.delete(KubernetesSecrets.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:145) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,190[aceSharedPool-3] [ERROR] [o.e.c.w.i.k.w.b.DeployBroker 147] - Brokers secret removal failed. Error: Your session has expired. Please login to Che again to get access to your OpenShift account org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesSecrets.delete(KubernetesSecrets.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:145) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,198[aceSharedPool-3] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.delete(KubernetesConfigsMaps.java:87) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:150) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,198[aceSharedPool-3] [ERROR] [o.e.c.w.i.k.w.b.DeployBroker 152] - Brokers config map removal failed. Error: Your session has expired. Please login to Che again to get access to your OpenShift account org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.delete(KubernetesConfigsMaps.java:87) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:150) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,198[aceSharedPool-3] [WARN ] [.i.k.KubernetesInternalRuntime 257] - Failed to start Kubernetes runtime of workspace workspace0kv0zlnibbphjd8l. org.eclipse.che.api.workspace.server.spi.InfrastructureException: Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:198) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.create(KubernetesConfigsMaps.java:70) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.DeployBroker.execute(DeployBroker.java:98) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.PrepareStorage.execute(PrepareStorage.java:76) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.brokerphases.ListenBrokerEvents.execute(ListenBrokerEvents.java:63) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.PluginBrokerManager.getTooling(PluginBrokerManager.java:129) at org.eclipse.che.workspace.infrastructure.kubernetes.wsplugins.SidecarToolingProvisioner.provision(SidecarToolingProvisioner.java:90) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.provisionWorkspace(KubernetesInternalRuntime.java:288) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:204) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,201[aceSharedPool-3] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.createOC(OpenShiftClientFactory.java:97) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftRoutes.delete(OpenShiftRoutes.java:84) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,205[aceSharedPool-3] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesServices.delete(KubernetesServices.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,208[aceSharedPool-3] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesDeployments.delete(KubernetesDeployments.java:875) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,212[aceSharedPool-3] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesSecrets.delete(KubernetesSecrets.java:86) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,215[aceSharedPool-3] [ERROR] [.IdentityProviderConfigFactory 192] - Cannot retrieve User OpenShift token from the 'openshift-v4' identity provider org.eclipse.che.api.core.BadRequestException: Invalid token. at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.doRequest(KeycloakServiceClient.java:199) at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.getIdentityProviderToken(KeycloakServiceClient.java:137) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.personalizeConfig(IdentityProviderConfigFactory.java:175) at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.buildConfig(IdentityProviderConfigFactory.java:169) at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.buildConfig(OpenShiftClientFactory.java:152) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.create(KubernetesClientFactory.java:94) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesConfigsMaps.delete(KubernetesConfigsMaps.java:87) at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:288) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,215[aceSharedPool-3] [WARN ] [.i.k.KubernetesInternalRuntime 268] - Failed to clean up namespace after workspace 'workspace0kv0zlnibbphjd8l' start failing. org.eclipse.che.api.workspace.server.spi.InfrastructureException: Error(s) occurs while cleaning up the namespace. Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account Your session has expired. Please login to Che again to get access to your OpenShift account at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.KubernetesNamespace.doRemove(KubernetesNamespace.java:302) at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.cleanUp(OpenShiftProject.java:157) at org.eclipse.che.workspace.infrastructure.kubernetes.RuntimeCleaner.cleanUp(RuntimeCleaner.java:56) at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInternalRuntime.internalStart(KubernetesInternalRuntime.java:266) at org.eclipse.che.api.workspace.server.spi.InternalRuntime.start(InternalRuntime.java:141) at org.eclipse.che.api.workspace.server.WorkspaceRuntimes$StartRuntimeTask.run(WorkspaceRuntimes.java:969) at org.eclipse.che.commons.lang.concurrent.CopyThreadLocalRunnable.run(CopyThreadLocalRunnable.java:38) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:44) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.eclipse.che.commons.observability.CountedThreadFactory.lambda$newThread$0(CountedThreadFactory.java:75) at java.base/java.lang.Thread.run(Thread.java:829) 2021-04-27 11:12:33,257[aceSharedPool-3] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 995] - Workspace 'admin:vertx-health-checks-4cyx2' with id 'workspace0kv0zlnibbphjd8l' start failed