Uploaded image for project: 'Red Hat CodeReady Workspaces'
  1. Red Hat CodeReady Workspaces
  2. CRW-2363

"crwctl server:deploy" command failed to redeploy CRW with DevWorkspace engine enabled because of subscription error

    XMLWordPrintable

Details

    • Workaround Exists
    • Hide

      1. Remove installed CRW project.
      1.2. Go to "openshift-operators" namespace.
      1.3. Uninstall CRW and DWO operators
      1.4. Delete next operators:

      • codeready-workspaces.openshift-operators (if present from 2.11 or 2.12 install from "latest" channel)
      • codeready-workspaces2.openshift-operators (if present from 2.12 install from "tech-preview-latest-all-namespaces" channel)
      • devworkspace-operator.openshift-operators

      1.5. Delete "controller.devfile.io" webhooks.
      1.6. Delete "devworkspace-webhook-server" deployment.
      1.7. Delete "devworkspace-webhookserver" service.
      1.8. Delete ConfigMap "*.devfile.io" and "**.org.eclipse.che".
      1.9. Delete "devworkspace-**" secrets.

      2. Deploy CRW with DevWorkspaces enabled.

      Show
      1. Remove installed CRW project. 1.2. Go to " openshift-operators " namespace. 1.3. Uninstall CRW and DWO operators 1.4. Delete next operators: codeready-workspaces.openshift-operators (if present from 2.11 or 2.12 install from "latest" channel) codeready-workspaces2.openshift-operators (if present from 2.12 install from "tech-preview-latest-all-namespaces" channel) devworkspace-operator.openshift-operators 1.5. Delete " controller.devfile.io " webhooks. 1.6. Delete " devworkspace-webhook-server " deployment. 1.7. Delete " devworkspace-webhookserver " service. 1.8. Delete ConfigMap " * .devfile.io " and " **.org.eclipse.che ". 1.9. Delete " devworkspace-** " secrets. 2. Deploy CRW with DevWorkspaces enabled.
    • Hide
      = Uninstalling {prod-short} fails when the {devworkspace} engine is enabled

      When the {devworkspace} engine is enabled, uninstalling {prod-short} fails, meaning:

      * The deletion process doesn't remove the {devworkspace} engine.

      .Workaround

      * In the `openshift-operators` namespace:

      . Uninstall the {prod-short} and {devworkspace} Operators
      . Delete the `controller.devfile.io` WebHooks.
      . Delete the `devworkspace-webhook-server` Deployment.
      . Delete the `devworkspace-webhookserver" Service.
      . Delete all `+**.devfile.io+` and `+**.org.eclipse.che+` ConfigMaps.
      . Delete all `+devworkspace-**+` Secrets.

      Show
      = Uninstalling {prod-short} fails when the {devworkspace} engine is enabled When the {devworkspace} engine is enabled, uninstalling {prod-short} fails, meaning: * The deletion process doesn't remove the {devworkspace} engine. .Workaround * In the `openshift-operators` namespace: . Uninstall the {prod-short} and {devworkspace} Operators . Delete the `controller.devfile.io` WebHooks. . Delete the `devworkspace-webhook-server` Deployment. . Delete the `devworkspace-webhookserver" Service. . Delete all `+**.devfile.io+` and `+**.org.eclipse.che+` ConfigMaps. . Delete all `+devworkspace-**+` Secrets.
    • Documented as Known Issue

    Description

      "crwctl server:delete" command doesn't cleanup 'openshift-operators' namespace:

      • doesn't remove DevWorkspace and CRW operators:
      • doesn't remove "controller.devfile.io" webhooks:

      Version: crwctl/2.12.0-CI-redhat.05e8a77 linux-x64 node-v12.22.3

      https://main-jenkins-csb-crwqe.apps.ocp4.prod.psi.redhat.com/job/CodereadyWorkspaces/job/e2e/job/basic/job/uninstall-crw/99/console

      "'crwctl delete' command log"
      + /mnt/hudson_workspace/workspace/CodereadyWorkspaces/e2e/basic/uninstall-crw/crwctl/bin/crwctl server:delete -n install-crw-using-crwctl --delete-namespace --yes --listr-renderer=verbose
       [15:48:52] Verify Kubernetes API [started]
       › Current Kubernetes context: 'default/api-cluster-2318-2318-sandbox453-opentlc-com:6443/admin'
       (node:9901) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
       [15:48:52] Verify Kubernetes API...OK [title changed]
       [15:48:52] Verify Kubernetes API...OK (it's OpenShift) [title changed]
       [15:48:52] Verify Kubernetes API...OK (it's OpenShift) [completed]
       [15:48:52] Delete oauthClientAuthorizations [started]
       [15:48:52] Delete oauthClientAuthorizations...OK [title changed]
       [15:48:52] Delete oauthClientAuthorizations...OK [completed]
       [15:48:52] Delete the Custom Resource of type checlusters.org.eclipse.che [started]
       [15:48:54] Delete the Custom Resource of type checlusters.org.eclipse.che...OK [title changed]
       [15:48:54] Delete the Custom Resource of type checlusters.org.eclipse.che...OK [completed]
       [15:48:54] Delete CRDs [started]
       [15:48:54] Delete CRDs...OK [title changed]
       [15:48:54] Delete CRDs...OK [completed]
       [15:48:54] Delete Roles and Bindings [started]
       [15:48:55] Delete Roles and Bindings...OK [title changed]
       [15:48:55] Delete Roles and Bindings...OK [completed]
       [15:48:55] Delete service accounts codeready-operator [started]
       [15:48:55] Delete service accounts codeready-operator...OK [title changed]
       [15:48:55] Delete service accounts codeready-operator...OK [completed]
       [15:48:55] Delete PVC codeready-operator [started]
       [15:48:55] Delete PVC codeready-operator...OK [title changed]
       [15:48:55] Delete PVC codeready-operator...OK [completed]
       [15:48:55] Check if OLM is pre-installed on the platform [started]
       [15:48:55] Check if OLM is pre-installed on the platform: true...OK [title changed]
       [15:48:55] Check if OLM is pre-installed on the platform: true...OK [completed]
       [15:48:55] Check if operator is installed in openshift-operators namespace [started]
       [15:48:56] Check if operator is installed in openshift-operators namespace...Found [title changed]
       [15:48:56] Check if operator is installed in openshift-operators namespace...Found [completed]
       [15:48:56] Delete(OLM) operator subscription codeready-subscription [started]
       [15:48:56] Delete(OLM) operator subscription codeready-subscription...OK [title changed]
       [15:48:56] Delete(OLM) operator subscription codeready-subscription...OK [completed]
       [15:48:56] Delete(OLM) CodeReady Workspaces cluster service versions [started]
       [15:48:56] Delete(OLM) CodeReady Workspaces cluster service versions...OK [title changed]
       [15:48:56] Delete(OLM) CodeReady Workspaces cluster service versions...OK [completed]
       [15:48:56] Delete(OLM) operator group codeready-operator-group [started]
       [15:48:56] Delete(OLM) operator group codeready-operator-group...OK [title changed]
       [15:48:56] Delete(OLM) operator group codeready-operator-group...OK [completed]
       [15:48:56] Delete(OLM) custom catalog source codeready-custom-catalog-source [started]
       [15:48:56] Delete(OLM) custom catalog source codeready-custom-catalog-source...OK [title changed]
       [15:48:56] Delete(OLM) custom catalog source codeready-custom-catalog-source...OK [completed]
       [15:48:56] Delete(OLM) nigthly catalog source eclipse-che-preview [started]
       [15:48:56] Delete(OLM) nigthly catalog source eclipse-che-preview...OK [title changed]
       [15:48:56] Delete(OLM) nigthly catalog source eclipse-che-preview...OK [completed]
       [15:48:56] Delete role prometheus-k8s [started]
       [15:48:56] Delete role prometheus-k8s...OK [title changed]
       [15:48:56] Delete role prometheus-k8s...OK [completed]
       [15:48:56] Delete role binding prometheus-k8s [started]
       [15:48:56] Delete role binding prometheus-k8s...OK [title changed]
       [15:48:56] Delete role binding prometheus-k8s...OK [completed]
       [15:48:56] Delete all deployments [started]
       [15:48:56] Delete all deployments...OK [title changed]
       [15:48:56] Delete all deployments...OK [completed]
       [15:48:56] Delete all services [started]
       [15:48:56] Delete all services...OK [title changed]
       [15:48:56] Delete all services...OK [completed]
       [15:48:56] Delete all routes [started]
       [15:48:56] Delete all routes...OK [title changed]
       [15:48:56] Delete all routes...OK [completed]
       [15:48:56] Delete configmaps for CodeReady Workspaces server and operator [started]
       [15:48:57] Delete configmaps for CodeReady Workspaces server and operator...OK [title changed]
       [15:48:57] Delete configmaps for CodeReady Workspaces server and operator...OK [completed]
       [15:48:57] Delete rolebindings che, che-workspace-exec and che-workspace-view [started]
       [15:48:57] Delete rolebindings che, che-workspace-exec and che-workspace-view...OK [title changed]
       [15:48:57] Delete rolebindings che, che-workspace-exec and che-workspace-view...OK [completed]
       [15:48:57] Delete service accounts che, che-workspace [started]
       [15:48:57] Delete service accounts che, che-workspace...OK [title changed]
       [15:48:57] Delete service accounts che, che-workspace...OK [completed]
       [15:48:57] Delete PVCs [started]
       [15:48:57] Delete PVCs...OK [title changed]
       [15:48:57] Delete PVCs...OK [completed]
       [15:48:57] Delete consoleLink che [started]
       [15:48:57] Delete consoleLink che...OK [title changed]
       [15:48:57] Delete consoleLink che...OK [completed]
       [15:48:57] Wait until CodeReady Workspaces Server pod is deleted [started]
       [15:48:57] Wait until CodeReady Workspaces Server pod is deleted...done. [title changed]
       [15:48:57] Wait until CodeReady Workspaces Server pod is deleted...done. [completed]
       [15:48:57] Wait until CodeReady Workspaces Dashboard pod is deleted [started]
       [15:48:58] Wait until CodeReady Workspaces Dashboard pod is deleted...done. [title changed]
       [15:48:58] Wait until CodeReady Workspaces Dashboard pod is deleted...done. [completed]
       [15:48:58] Wait until Keycloak pod is deleted [started]
       [15:48:58] Wait until Keycloak pod is deleted...done. [title changed]
       [15:48:58] Wait until Keycloak pod is deleted...done. [completed]
       [15:48:58] Wait until PostgreSQL pod is deleted [started]
       [15:48:58] Wait until PostgreSQL pod is deleted...done. [title changed]
       [15:48:58] Wait until PostgreSQL pod is deleted...done. [completed]
       [15:48:58] Wait until Devfile Registry pod is deleted [started]
       [15:48:58] Wait until Devfile Registry pod is deleted...done. [title changed]
       [15:48:58] Wait until Devfile Registry pod is deleted...done. [completed]
       [15:48:58] Wait until Plug-in Registry pod is deleted [started]
       [15:48:58] Wait until Plug-in Registry pod is deleted...done. [title changed]
       [15:48:58] Wait until Plug-in Registry pod is deleted...done. [completed]
       [15:48:58] Uninstall DevWorkspace Controller and DevWorkspace Che Controller [started]
       [15:48:58] Delete all DevWorkspace Controller deployments [started]
       [15:48:58] Delete all DevWorkspace Controller deployments...OK [title changed]
       [15:48:58] Delete all DevWorkspace Controller deployments...OK [completed]
       [15:48:58] Delete all DevWorkspace Controller services [started]
       [15:48:58] Delete all DevWorkspace Controller services...OK [title changed]
       [15:48:58] Delete all DevWorkspace Controller services...OK [completed]
       [15:48:58] Delete all DevWorkspace Controller routes [started]
       [15:48:58] Delete all DevWorkspace Controller routes...OK [title changed]
       [15:48:58] Delete all DevWorkspace Controller routes...OK [completed]
       [15:48:58] Delete DevWorkspace Controller configmaps [started]
       [15:48:58] Delete DevWorkspace Controller configmaps...OK [title changed]
       [15:48:58] Delete DevWorkspace Controller configmaps...OK [completed]
       [15:48:58] Delete DevWorkspace Controller ClusterRoleBindings [started]
       [15:48:58] Delete DevWorkspace Controller ClusterRoleBindings...OK [title changed]
       [15:48:58] Delete DevWorkspace Controller ClusterRoleBindings...OK [completed]
       [15:48:58] Delete DevWorkspace Controller role [started]
       [15:48:59] Delete DevWorkspace Controller role...OK [title changed]
       [15:48:59] Delete DevWorkspace Controller role...OK [completed]
       [15:48:59] Delete DevWorkspace Controller roleBinding [started]
       [15:48:59] Delete DevWorkspace Controller roleBinding...OK [title changed]
       [15:48:59] Delete DevWorkspace Controller roleBinding...OK [completed]
       [15:48:59] Delete DevWorkspace Controller cluster roles [started]
       [15:48:59] Delete DevWorkspace Controller cluster roles...OK [title changed]
       [15:48:59] Delete DevWorkspace Controller cluster roles...OK [completed]
       [15:48:59] Delete DevWorkspace Controller service account [started]
       [15:48:59] Delete DevWorkspace Controller service account...OK [title changed]
       [15:48:59] Delete DevWorkspace Controller service account...OK [completed]
       [15:48:59] Delete DevWorkspace Controller self-signed certificates [started]
       [15:48:59] Delete DevWorkspace Controller self-signed certificates...OK [title changed]
       [15:48:59] Delete DevWorkspace Controller self-signed certificates...OK [completed]
       [15:48:59] Delete DevWorkspace Controller webhooks configurations [started]
       [15:48:59] Delete DevWorkspace Controller webhooks configurations ...OK [title changed]
       [15:48:59] Delete DevWorkspace Controller webhooks configurations ...OK [completed]
       [15:48:59] Delete DevWorkspace Controller CRDs [started]
       [15:49:00] Delete DevWorkspace Controller CRDs...OK [title changed]
       [15:49:00] Delete DevWorkspace Controller CRDs...OK [completed]
       [15:49:00] Uninstall DevWorkspace Controller and DevWorkspace Che Controller [completed]
       [15:49:00] Delete namespace install-crw-using-crwctl [started]
       [15:49:00] Delete namespace install-crw-using-crwctl [completed]
       Command server:delete has completed successfully in 00:08.
      

      As a result, "crwctl server:deploy" command executed after removal, had failed:
      https://main-jenkins-csb-crwqe.apps.ocp4.prod.psi.redhat.com/job/CodereadyWorkspaces/job/e2e/job/basic/job/install-crw-using-crwctl/1509/console

      "'crwctl server:deploy' command log"
       + crwctl/bin/crwctl server:deploy --no-olm-suggested-namespace --catalog-source-name=crw-2-12-osbs-tech-preview-latest-all-namespaces --catalog-source-namespace=openshift-marketplace --package-manifest-name=codeready-workspaces2 --olm-channel=tech-preview-latest-all-namespaces --listr-renderer=verbose -n install-crw-using-crwctl
       › Installer type is set to: 'olm'
       [15:19:32] Verify Kubernetes API [started]
       › Current Kubernetes context: 'default/api-cluster-2318-2318-sandbox453-opentlc-com:6443/admin'
       [15:19:32] Verify Kubernetes API...OK [title changed]
       [15:19:32] Verify Kubernetes API...OK (it's OpenShift) [title changed]
       [15:19:32] Verify Kubernetes API...OK (it's OpenShift) [completed]
       [15:19:32]   Looking for an already existing CodeReady Workspaces instance [started]
       [15:19:32] Verify if CodeReady Workspaces is deployed into namespace "install-crw-using-crwctl" [started]
       [15:19:32] Verify if CodeReady Workspaces is deployed into namespace "install-crw-using-crwctl"...it is not [title changed]
       [15:19:32] Verify if CodeReady Workspaces is deployed into namespace "install-crw-using-crwctl"...it is not [completed]
       [15:19:32]   Looking for an already existing CodeReady Workspaces instance [completed]
       [15:19:32]   Openshift preflight checklist [started]
       [15:19:32] Verify if oc is installed [started]
       [15:19:32] Verify if oc is installed...done. [title changed]
       [15:19:32] Verify if oc is installed...done. [completed]
       [15:19:32] Verify if openshift is running [started]
       [15:19:33] Verify if openshift is running...done. [title changed]
       [15:19:33] Verify if openshift is running...done. [completed]
       [15:19:33] Check OpenShift version [started]
       [15:19:33] Check OpenShift version: 4.x [title changed]
       [15:19:33] Check OpenShift version: 4.x [completed]
       [15:19:33] Check Kubernetes version [started]
       [15:19:33] Check Kubernetes version: Unknown. [title changed]
       [15:19:33] Check Kubernetes version: Unknown. [completed]
       [15:19:33]   Openshift preflight checklist [completed]
       [15:19:33] Following CodeReady Workspaces logs [started]
       [15:19:33] Start following Operator logs [started]
       [15:19:33] Start following Operator logs...done [title changed]
       [15:19:33] Start following Operator logs...done [completed]
       [15:19:33] Start following CodeReady Workspaces Server logs [started]
       [15:19:33] Start following CodeReady Workspaces Server logs...done [title changed]
       [15:19:33] Start following CodeReady Workspaces Server logs...done [completed]
       [15:19:33] Start following PostgreSQL logs [started]
       [15:19:33] Start following PostgreSQL logs...done [title changed]
       [15:19:33] Start following PostgreSQL logs...done [completed]
       [15:19:33] Start following Keycloak logs [started]
       [15:19:33] Start following Keycloak logs...done [title changed]
       [15:19:33] Start following Keycloak logs...done [completed]
       [15:19:33] Start following Plug-in Registry logs [started]
       [15:19:33] Start following Plug-in Registry logs...done [title changed]
       [15:19:33] Start following Plug-in Registry logs...done [completed]
       [15:19:33] Start following Devfile Registry logs [started]
       [15:19:33] Start following Devfile Registry logs...done [title changed]
       [15:19:33] Start following Devfile Registry logs...done [completed]
       [15:19:33] Start following CodeReady Workspaces Dashboard logs [started]
       [15:19:33] Start following CodeReady Workspaces Dashboard logs...done [title changed]
       [15:19:33] Start following CodeReady Workspaces Dashboard logs...done [completed]
       [15:19:33] Start following namespace events [started]
       [15:19:33] Start following namespace events...done [title changed]
       [15:19:33] Start following namespace events...done [completed]
       [15:19:33] Following CodeReady Workspaces logs [completed]
       [15:19:33]   Running Olm installaion CodeReady Workspaces [started]
       [15:19:33] Check if OLM is pre-installed on the platform [started]
       [15:19:33] Check if OLM is pre-installed on the platform...done. [title changed]
       [15:19:33] Check if OLM is pre-installed on the platform...done. [completed]
       [15:19:33] Create Namespace (install-crw-using-crwctl) [started]
       [15:19:34] Create Namespace (install-crw-using-crwctl)...Done. [title changed]
       [15:19:34] Create Namespace (install-crw-using-crwctl)...Done. [completed]
       [15:19:34] Configure context information [started]
       [15:19:34] Configure context information...done. [title changed]
       [15:19:34] Configure context information...done. [completed]
       [15:19:34] Create operator subscription [started]
       [15:19:34] Create operator subscription...created new one. [title changed]
       [15:19:34] Create operator subscription...created new one. [completed]
       [15:19:34] Wait while subscription is ready [started]
       [15:29:34] Wait while subscription is ready [failed]
       [15:29:34]  undefined
       [15:29:34]   Running Olm installaion CodeReady Workspaces [failed]
       [15:29:34]  Cannot create property 'context' on string 'Timeout reached while waiting for "codeready-subscription" subscription is ready.'
           Error: Command server:deploy failed. Error log: 
           /home/hudson/.cache/crwctl/error.log.
      

      Update: it was impossible to remove "devworkspace-operator.openshift-operators" and "devworkspace..." secrets manually from "openshift-operators" project:

       

      Update 2: CRW reinstalled from OperatorHub, wasn't able to create workspace because of error:

      Failed to create a new workspace from the devfile, reason: create not allowed while custom resource definition is terminating
      

      Attachments

        Activity

          People

            abazko Anatolii Bazko
            dnochevn Dmytro Nochevnov
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: