Uploaded image for project: 'Red Hat OpenShift Dev Spaces (formerly CodeReady Workspaces) '
  1. Red Hat OpenShift Dev Spaces (formerly CodeReady Workspaces)
  2. CRW-2182

Persistent volume claim not terminating after deleting workspaces

XMLWordPrintable

    • False
    • False
    • undefined
    • Hide

      1. Install CRW 2.10.1 on an RHPDS OpenShift 4.7 Workshop (Training)

      2. Create a new CheCluster, and log in to the provided CRW url withthe opentlc-mgr user

      3. Create and stop three Python workspaces

      4. On the workspaces page, select all workspaces and delete them (see threeWorkspaces.PNG)

      5. Navigate to PersistentVolumeClaims in the opentlc-mgr-codeready namespace in OpenShift

      6. Observe that the claim-che-workspace is in a persistent Terminating state (see pvcTerminating.PNG)

      7. Attempt to create a new Workspace in CRW

      8. Navigate to Pods in the opentlc-mgr-codeready namespace and observe one of two behaviors:

      a. the mkdir pod is stuck Pending with event stating no nodes available (See mkdirPodStuckPending.PNG)

      b. the che-plugin-broker pod is stuck Pending with the same event (See che-plugin-brokerPensing.PNG)

      9. As a workaround, delete the new workspace in CRW that will not start

      10. navigate to the claim-che-workspace PVC in the opentlc-mgr-codeready namespace and remove the value in metadata.finalizers (See removeFinalizers.PNG and finalizersRemoved.PNG)

      11. wait for the PVC to finish terminating

      12. navigate to Pods in the opentlc-mgr-codeready namespace and remove any pods that are in a pending state

      13. Recreate the workspace in CRW 

      Show
      1. Install CRW 2.10.1 on an RHPDS OpenShift 4.7 Workshop (Training) 2. Create a new CheCluster, and log in to the provided CRW url withthe opentlc-mgr user 3. Create and stop three Python workspaces 4. On the workspaces page, select all workspaces and delete them (see threeWorkspaces.PNG) 5. Navigate to PersistentVolumeClaims in the opentlc-mgr-codeready namespace in OpenShift 6. Observe that the claim-che-workspace is in a persistent Terminating state (see pvcTerminating.PNG) 7. Attempt to create a new Workspace in CRW 8. Navigate to Pods in the opentlc-mgr-codeready namespace and observe one of two behaviors: a. the mkdir pod is stuck Pending with event stating no nodes available (See mkdirPodStuckPending.PNG) b. the che-plugin-broker pod is stuck Pending with the same event (See che-plugin-brokerPensing.PNG) 9. As a workaround, delete the new workspace in CRW that will not start 10. navigate to the claim-che-workspace PVC in the opentlc-mgr-codeready namespace and remove the value in metadata.finalizers (See removeFinalizers.PNG and finalizersRemoved.PNG) 11. wait for the PVC to finish terminating 12. navigate to Pods in the opentlc-mgr-codeready namespace and remove any pods that are in a pending state 13. Recreate the workspace in CRW 

      Frequently when deleting the one remaining workspace for a user, CRW attempts to terminate the PVC claim-che-workspace. Frequently the claim will get stuck in a Terminating state. This appears to happen more reliably if you have multiple workspaces and delete them all at the same time

        1. che-plugin-brokerPensing.PNG
          che-plugin-brokerPensing.PNG
          34 kB
        2. finalizersRemoved.PNG
          finalizersRemoved.PNG
          32 kB
        3. pvcTerminating.PNG
          pvcTerminating.PNG
          22 kB
        4. mkdirPodStuckPending.PNG
          mkdirPodStuckPending.PNG
          31 kB
        5. threeWorkspaces.PNG
          threeWorkspaces.PNG
          28 kB
        6. removeFinalizers.PNG
          removeFinalizers.PNG
          29 kB

              ibuziuk@redhat.com Ilya Buziuk
              myarbrou@redhat.com Michael Yarbrough
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: