Loading...

XML

Word

Printable

Type: Bug
Resolution: Done
Priority: Major
Fix Version/s: 3.3.0.GA
Affects Version/s: 3.2.0.GA
Component/s: deploy: cli, install, machine-exec, operator
Labels:
None

Blocked:
False
Blocked Reason:
None
Ready:
False
Affects:

Release Notes
Release Note Text:

Hide
= Redeploying the DevWorkspace Operator might fail

Currently, redeploying the DevWorkspace Operator fails after running the `dsc server:delete --delete-all` command.

.Workaround

. Delete the remaining DevWorkspace Operator resources:
+
[source]
----
workspaces=$(oc get devworkspaces --all-namespaces | awk 'NR>1 {print $1}') \
  && for workspace in $workspaces ; do \
       oc patch devworkspaces/$workspace -p '{ "metadata": { "finalizers": null }}' --type merge || true; \
       oc delete devworkspaces/$workspace || true; \
     done \
  && oc delete crd/devworkspaces.workspace.devfile.io
----
+
NOTE: If the `oc delete crd/devworkspaces.workspace.devfile.io` command fails, you must deep clean your OpenShift cluster.

. Redeploy the DevWorkspace Operator.

Show
= Redeploying the DevWorkspace Operator might fail Currently, redeploying the DevWorkspace Operator fails after running the `dsc server:delete --delete-all` command. .Workaround . Delete the remaining DevWorkspace Operator resources: + [source] ---- workspaces=$(oc get devworkspaces --all-namespaces | awk 'NR>1 {print $1}') \   && for workspace in $workspaces ; do \        oc patch devworkspaces/$workspace -p '{ "metadata": { "finalizers": null }}' --type merge || true; \        oc delete devworkspaces/$workspace || true; \      done \   && oc delete crd/devworkspaces.workspace.devfile.io ---- + NOTE: If the `oc delete crd/devworkspaces.workspace.devfile.io` command fails, you must deep clean your OpenShift cluster. . Redeploy the DevWorkspace Operator.
Release Note Type:
Known Issue
Release Note Status:
Done
Workaround Description:
Hide

1. Delete remaining DevWorkspace Operator resources:

workspaces=$(oc get devworkspaces --all-namespaces | awk 'NR>1 {print $1}') \ && for workspace in $workspaces ; do \ oc patch devworkspaces/$workspace -p '{ "metadata": { "finalizers": null }}' --type merge || true; \ oc delete devworkspaces/$workspace || true; \ done \ && oc delete crd/devworkspaces.workspace.devfile.io

2. Redeploy DevWorkspace Operator.

Note: if "oc delete crd/devworkspaces.workspace.devfile.io" failed, you need deep clean up of cluster.
Show
1. Delete remaining DevWorkspace Operator resources: workspaces=$(oc get devworkspaces --all-namespaces | awk 'NR>1 {print $1}' ) \ && for workspace in $workspaces ; do \ oc patch devworkspaces/$workspace -p '{ "metadata" : { "finalizers" : null }}' --type merge || true ; \ oc delete devworkspaces/$workspace || true ; \ done \ && oc delete crd/devworkspaces.workspace.devfile.io 2. Redeploy DevWorkspace Operator. Note : if "oc delete crd/devworkspaces.workspace.devfile.io" failed, you need deep clean up of cluster.

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

Test scenario:
1. Open OCP 4.11
2. Install DWO 0.16.0.RC-10-14
3. Install DS 3.2.0.RC-10-12_163
4. Create new workspaces.
5. Delete DS 3.2.0 using "dsc server:delete --delete-all" command (dsc/3.2.0-CI-ce41-redhat.a05925f linux-x64 node-v16.13.2)
6. Try to install DWO 0.16.0.RC-10-14 again.

Observed result:
pending DWO installation

error validating existing CRs against new CRD's schema for "devworkspaces.workspace.devfile.io": error listing resources in GroupVersionResource schema.GroupVersionResource

Unknown macro: {Group}

: conversion webhook for workspace.devfile.io/v1alpha2, Kind=DevWorkspace failed: Post "https://devworkspace-controller-manager-service.openshift-operators.svc:443/convert?timeout=30s": no endpoints available for service "devworkspace-controller-manager-service"

clusterserviceversion-devworkspace-operator.v0.16.0.yaml

dsc server:delete command logs

 + /mnt/hudson_workspace/workspace/Testing/e2e/basic/install-ds/dsc/bin/dsc server:delete --delete-all --yes --listr-renderer=verbose
 [10:45:10] Verify Kubernetes API [started]
 › Current Kubernetes context: 'default/api-ocp411-aws-crw-qe-com:6443/admin'
 [10:45:10] Verify Kubernetes API...[OK] [title changed]
 [10:45:10] Verify Kubernetes API...[OK] [completed]
 [10:45:10] Uninstall Dev Workspace Operator [started]
 [10:45:11] Delete Custom Resources [started]
 [10:45:11] Delete workspace.devfile.io/v1alpha1 resources [started]
 [10:45:11] Delete workspace.devfile.io/v1alpha1 resources...[Failed: conversion webhook for workspace.devfile.io/v1alpha2, Kind=DevWorkspace failed: Post "https://devworkspace-controller-manager-service.openshift-operators.svc:443/convert?timeout=30s": service "devworkspace-controller-manager-service" not found] [title changed]
 [10:45:11] Delete workspace.devfile.io/v1alpha1 resources...[Failed: conversion webhook for workspace.devfile.io/v1alpha2, Kind=DevWorkspace failed: Post "https://devworkspace-controller-manager-service.openshift-operators.svc:443/convert?timeout=30s": service "devworkspace-controller-manager-service" not found] [completed]
 [10:45:11] Delete controller.devfile.io/v1alpha1 resources [started]
 [10:45:24] Delete controller.devfile.io/v1alpha1 resources...[Ok] [title changed]
 [10:45:24] Delete controller.devfile.io/v1alpha1 resources...[Ok] [completed]
 [10:45:24] Delete Custom Resources [completed]
 [10:45:24] Delete operator resources [started]
 [10:45:24] Delete WebhookConfigurations controller.devfile.io [started]
 [10:45:25] Delete WebhookConfigurations controller.devfile.io ...[Ok] [title changed]
 [10:45:25] Delete WebhookConfigurations controller.devfile.io ...[Ok] [completed]
 [10:45:25] Delete CRDs [started]
 [10:45:26] Delete CRDs...[Ok] [title changed]
 [10:45:26] Delete CRDs...[Ok] [completed]
 [10:45:26] Delete Deployments [started]
 [10:45:27] Delete Deployments...[Ok] [title changed]
 [10:45:27] Delete Deployments...[Ok] [completed]
 [10:45:27] Delete Services [started]
 [10:45:27] Delete Services...[Ok] [title changed]
 [10:45:27] Delete Services...[Ok] [completed]
 [10:45:27] Delete Secrets [started]
 [10:45:28] Delete Secrets...[Ok] [title changed]
 [10:45:28] Delete Secrets...[Ok] [completed]
 [10:45:28] Delete RoleBindings [started]
 [10:45:28] Delete RoleBindings...[Ok] [title changed]
 [10:45:28] Delete RoleBindings...[Ok] [completed]
 [10:45:28] Delete Roles [started]
 [10:45:29] Delete Roles...[Ok] [title changed]
 [10:45:29] Delete Roles...[Ok] [completed]
 [10:45:29] Delete ClusterRoleBindings [started]
 [10:45:29] Delete ClusterRoleBindings...[Ok] [title changed]
 [10:45:29] Delete ClusterRoleBindings...[Ok] [completed]
 [10:45:29] Delete ClusterRoles [started]
 [10:45:31] Delete ClusterRoles...[Ok] [title changed]
 [10:45:31] Delete ClusterRoles...[Ok] [completed]
 [10:45:31] Delete ServiceAccounts [started]
 [10:45:31] Delete ServiceAccounts...[Ok] [title changed]
 [10:45:31] Delete ServiceAccounts...[Ok] [completed]
 [10:45:31] Delete Issuer devworkspace-controller-selfsigned-issuer [started]
 [10:45:31] Delete Issuer devworkspace-controller-selfsigned-issuer...[Ok] [title changed]
 [10:45:31] Delete Issuer devworkspace-controller-selfsigned-issuer...[Ok] [completed]
 [10:45:31] Delete Certificate devworkspace-controller-serving-cert [started]
 [10:45:32] Delete Certificate devworkspace-controller-serving-cert...[Ok] [title changed]
 [10:45:32] Delete Certificate devworkspace-controller-serving-cert...[Ok] [completed]
 [10:45:32] Delete operator resources [completed]
 [10:45:32] Uninstall Dev Workspace Operator [completed]
 [10:45:32] Uninstall Red Hat OpenShift Dev Spaces Operator [started]
 [10:45:32] Delete Custom Resources [started]
 [10:45:32] Delete org.eclipse.che/v2 resources [started]
 [10:45:33] Delete org.eclipse.che/v2 resources...[Ok] [title changed]
 [10:45:33] Delete org.eclipse.che/v2 resources...[Ok] [completed]
 [10:45:33] Delete Custom Resources [completed]
 [10:45:33] Delete OLM resources [started]
 [10:45:33] Check if OLM is pre-installed on the platform [started]
 [10:45:33] Check if OLM is pre-installed on the platform...[Found] [title changed]
 [10:45:33] Check if OLM is pre-installed on the platform...[Found] [completed]
 [10:45:33] Delete Subscription [started]
 [10:45:33] Delete Subscription...[Ok] [title changed]
 [10:45:33] Delete Subscription...[Ok] [completed]
 [10:45:33] Delete ClusterServiceVersion [started]
 [10:45:34] Delete ClusterServiceVersion...[Ok] [title changed]
 [10:45:34] Delete ClusterServiceVersion...[Ok] [completed]
 [10:45:34] Delete CatalogSources [started]
 [10:45:34] Delete CatalogSources...[Ok] [title changed]
 [10:45:34] Delete CatalogSources...[Ok] [completed]
 [10:45:34] Delete Role prometheus-k8s [started]
 [10:45:35] Delete Role prometheus-k8s...[Ok] [title changed]
 [10:45:35] Delete Role prometheus-k8s...[Ok] [completed]
 [10:45:35] Delete RoleBinding prometheus-k8s [started]
 [10:45:35] Delete RoleBinding prometheus-k8s...[Ok] [title changed]
 [10:45:35] Delete RoleBinding prometheus-k8s...[Ok] [completed]
 [10:45:35] Delete OLM resources [completed]
 [10:45:35] Delete operator resources [started]
 [10:45:35] Delete ValidatingWebhookConfiguration org.eclipse.che [started]
 [10:45:35] Delete ValidatingWebhookConfiguration org.eclipse.che...[Ok] [title changed]
 [10:45:35] Delete ValidatingWebhookConfiguration org.eclipse.che...[Ok] [completed]
 [10:45:35] Delete CRDs [started]
 [10:45:35] Delete CRDs...[Ok] [title changed]
 [10:45:35] Delete CRDs...[Ok] [completed]
 [10:45:35] Delete Deployments [started]
 [10:45:35] Delete Deployments...[Ok] [title changed]
 [10:45:35] Delete Deployments...[Ok] [completed]
 [10:45:35] Delete Services [started]
 [10:45:36] Delete Services...[Ok] [title changed]
 [10:45:36] Delete Services...[Ok] [completed]
 [10:45:36] Delete Ingresses [started]
 [10:45:36] Delete Ingresses...[Ok] [title changed]
 [10:45:36] Delete Ingresses...[Ok] [completed]
 [10:45:36] Delete Secrets [started]
 [10:45:36] Delete Secrets...[Ok] [title changed]
 [10:45:36] Delete Secrets...[Ok] [completed]
 [10:45:36] Delete ConfigMaps [started]
 [10:45:37] Delete ConfigMaps...[Ok] [title changed]
 [10:45:37] Delete ConfigMaps...[Ok] [completed]
 [10:45:37] Delete RoleBindings [started]
 [10:45:41] Delete RoleBindings...[Ok] [title changed]
 [10:45:41] Delete RoleBindings...[Ok] [completed]
 [10:45:41] Delete Roles [started]
 [10:45:41] Delete Roles...[Ok] [title changed]
 [10:45:41] Delete Roles...[Ok] [completed]
 [10:45:41] Delete ClusterRoles [started]
 [10:45:43] Delete ClusterRoles...[Ok] [title changed]
 [10:45:43] Delete ClusterRoles...[Ok] [completed]
 [10:45:43] Delete ClusterRoleBindings [started]
 [10:45:44] Delete ClusterRoleBindings...[Ok] [title changed]
 [10:45:44] Delete ClusterRoleBindings...[Ok] [completed]
 [10:45:44] Delete ServiceAccounts [started]
 [10:45:45] Delete ServiceAccounts...[Ok] [title changed]
 [10:45:45] Delete ServiceAccounts...[Ok] [completed]
 [10:45:45] Delete Issuer devspaces-operator-selfsigned-issuer [started]
 [10:45:45] Delete Issuer devspaces-operator-selfsigned-issuer...[Ok] [title changed]
 [10:45:45] Delete Issuer devspaces-operator-selfsigned-issuer...[Ok] [completed]
 [10:45:45] Delete Certificate devspaces-operator-serving-cert [started]
 [10:45:45] Delete Certificate devspaces-operator-serving-cert...[Ok] [title changed]
 [10:45:45] Delete Certificate devspaces-operator-serving-cert...[Ok] [completed]
 [10:45:45] Delete PVCs [started]
 [10:45:45] Delete PVCs...[Ok] [title changed]
 [10:45:45] Delete PVCs...[Ok] [completed]
 [10:45:45] Delete ConsoleLink che [started]
 [10:45:46] Delete ConsoleLink che...[Ok] [title changed]
 [10:45:46] Delete ConsoleLink che...[Ok] [completed]
 [10:45:46] Delete operator resources [completed]
 [10:45:46] Wait until all pods are deleted [started]
 [10:45:46] Che Server pod [started]
 [10:45:46] Che Server pod...[Ok] [title changed]
 [10:45:46] Che Server pod...[Ok] [completed]
 [10:45:46] Dashboard pod [started]
 [10:45:46] Dashboard pod...[Ok] [title changed]
 [10:45:46] Dashboard pod...[Ok] [completed]
 [10:45:46] PostgreSQL pod [started]
 [10:45:46] PostgreSQL pod...[Ok] [title changed]
 [10:45:46] PostgreSQL pod...[Ok] [completed]
 [10:45:46] Devfile Registry pod [started]
 [10:45:46] Devfile Registry pod...[Ok] [title changed]
 [10:45:46] Devfile Registry pod...[Ok] [completed]
 [10:45:46] Plug-in Registry [started]
 [10:45:47] Plug-in Registry...[Ok] [title changed]
 [10:45:47] Plug-in Registry...[Ok] [completed]
 [10:45:47] Wait until all pods are deleted [completed]
 [10:45:47] Delete namespace openshift-devspaces [started]
 [10:45:47] Delete namespace openshift-devspaces [completed]
 [10:45:47] Uninstall Red Hat OpenShift Dev Spaces Operator [completed]
 Command server:delete has completed successfully in 00:37.

Test pipeline: https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/Testing/job/e2e/job/basic/job/install-ds/1072/console

Case 2:

https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/Testing/job/e2e/job/basic/job/install-ds/1089/console

Remained DWO resources:

$ oc get devworkspaces -n admin-devspaces
NAME                    DEVWORKSPACE ID             PHASE     INFO
java-gradle             workspaced96226f01e8b4d4c   Failed    Container tools has state ImagePullBackOff
java-lombok             workspaceda2e2a739a7444b4   Running   https://devspaces.apps.ocp411.crw-qe.com/workspaceda2e2a739a7444b4/idea-rhel8/8887/?backgroundColor=434343&wss
jboss-eap-quickstarts   workspacebfe11afe979e4e8a   Failed    devworkspace failed to progress past phase 'Starting' for longer than timeout (10m)

$ oc get crd | grep devworkspace
devworkspaces.workspace.devfile.io                                2022-10-12T16:22:18Z

Update: the issue hasn't been reproduced on OCP 4.10: https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/Testing/job/e2e/job/basic/job/install-ds/1078/console

- - Sort By Name
  - Sort By Date
  - Ascending
  - Descending
  - Thumbnails
  - List
  - Download All

clusterserviceversion-devworkspace-operator.v0.16.0.yaml
2022/10/14 3:16 PM
38 kB
Dmytro Nochevnov
Screenshot from 2022-10-14 18-08-51.png
2022/10/14 3:15 PM
171 kB
Dmytro Nochevnov

is related to

CRW-3203 [RN] Remove workspaces and CRDs when removing dev spaces

Closed

relates to

CRW-3536 "dcs server:delete --delete-all" command didn't remove remaining "devworkspaceoperatorconfig" resource

Closed

mentioned on

Merge request - Create 'admin-devspaces' namespace in api-tests pipeline; use 'dsc server:delete --delete-all' command

Merge request - Don't use '--delete-all' parameter in test pipelines with special openshift-devspaces project configuration

Assignee:: Anatolii Bazko

Reporter:: Dmytro Nochevnov

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Created:: 2022/10/14 3:18 PM

Updated:: 2022/11/17 11:39 PM

Resolved:: 2022/11/17 11:38 PM

Details

Description

Case 2:

Attachments

Attachments

Issue Links

Easy Agile Planning Poker

Activity

People

Dates