Uploaded image for project: 'Red Hat OpenShift Dev Spaces (formerly CodeReady Workspaces) '
  1. Red Hat OpenShift Dev Spaces (formerly CodeReady Workspaces)
  2. CRW-2182

Persistent volume claim not terminating after deleting workspaces

XMLWordPrintable

    • False
    • False
    • undefined
    • Hide

      1. Install CRW 2.10.1 on an RHPDS OpenShift 4.7 Workshop (Training)

      2. Create a new CheCluster, and log in to the provided CRW url withthe opentlc-mgr user

      3. Create and stop three Python workspaces

      4. On the workspaces page, select all workspaces and delete them (see threeWorkspaces.PNG)

      5. Navigate to PersistentVolumeClaims in the opentlc-mgr-codeready namespace in OpenShift

      6. Observe that the claim-che-workspace is in a persistent Terminating state (see pvcTerminating.PNG)

      7. Attempt to create a new Workspace in CRW

      8. Navigate to Pods in the opentlc-mgr-codeready namespace and observe one of two behaviors:

      a. the mkdir pod is stuck Pending with event stating no nodes available (See mkdirPodStuckPending.PNG)

      b. the che-plugin-broker pod is stuck Pending with the same event (See che-plugin-brokerPensing.PNG)

      9. As a workaround, delete the new workspace in CRW that will not start

      10. navigate to the claim-che-workspace PVC in the opentlc-mgr-codeready namespace and remove the value in metadata.finalizers (See removeFinalizers.PNG and finalizersRemoved.PNG)

      11. wait for the PVC to finish terminating

      12. navigate to Pods in the opentlc-mgr-codeready namespace and remove any pods that are in a pending state

      13. Recreate the workspace in CRW 

      Show
      1. Install CRW 2.10.1 on an RHPDS OpenShift 4.7 Workshop (Training) 2. Create a new CheCluster, and log in to the provided CRW url withthe opentlc-mgr user 3. Create and stop three Python workspaces 4. On the workspaces page, select all workspaces and delete them (see threeWorkspaces.PNG) 5. Navigate to PersistentVolumeClaims in the opentlc-mgr-codeready namespace in OpenShift 6. Observe that the claim-che-workspace is in a persistent Terminating state (see pvcTerminating.PNG) 7. Attempt to create a new Workspace in CRW 8. Navigate to Pods in the opentlc-mgr-codeready namespace and observe one of two behaviors: a. the mkdir pod is stuck Pending with event stating no nodes available (See mkdirPodStuckPending.PNG) b. the che-plugin-broker pod is stuck Pending with the same event (See che-plugin-brokerPensing.PNG) 9. As a workaround, delete the new workspace in CRW that will not start 10. navigate to the claim-che-workspace PVC in the opentlc-mgr-codeready namespace and remove the value in metadata.finalizers (See removeFinalizers.PNG and finalizersRemoved.PNG) 11. wait for the PVC to finish terminating 12. navigate to Pods in the opentlc-mgr-codeready namespace and remove any pods that are in a pending state 13. Recreate the workspace in CRW 

      Frequently when deleting the one remaining workspace for a user, CRW attempts to terminate the PVC claim-che-workspace. Frequently the claim will get stuck in a Terminating state. This appears to happen more reliably if you have multiple workspaces and delete them all at the same time

        1. che-plugin-brokerPensing.PNG
          34 kB
          Michael Yarbrough
        2. finalizersRemoved.PNG
          32 kB
          Michael Yarbrough
        3. mkdirPodStuckPending.PNG
          31 kB
          Michael Yarbrough
        4. pvcTerminating.PNG
          22 kB
          Michael Yarbrough
        5. removeFinalizers.PNG
          29 kB
          Michael Yarbrough
        6. threeWorkspaces.PNG
          28 kB
          Michael Yarbrough

              ibuziuk@redhat.com Ilya Buziuk
              myarbrou@redhat.com Michael Yarbrough
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: