Uploaded image for project: 'Red Hat OpenShift Data Science'
  1. Red Hat OpenShift Data Science
  2. RHODS-4505

GPU namespace stuck in terminating status after addon uninstall

XMLWordPrintable

    • False
    • None
    • False
    • Hide

      all the resources created by the operator must be deleted when uninstall is triggered

      Show
      all the resources created by the operator must be deleted when uninstall is triggered
    • No
    • No
    • No
    • None
    • RHODS 1.14
    • Medium

      Description of problem:

      After I triggered the GPU addon uninstall process, the namespace "redhat-nvidia-gpu-addon" is stuck under "terminating" status with the following message:

       

      Some content in the namespace has finalizers remaining:        
          foreground-deletion in 1 resource instances, nvidia-gpu-addon in 1        
          resource instances

      All the pods have been deleted and OCM UI reports the Addon as "uninstalled".

       

      The CRs that are blocking the uninstall process are:

      • ocp-gpu-addon (NodeFeatureDiscovery)
      • nvidia-gpu-addon (GPUAddon)

      Prerequisites (if any, like setup, operators/versions):

      Install RHODS

      Install Nvidia GPU Addon

      Steps to Reproduce

      1. Go to OCM (https://console.redhat.com/)
      2. select your cluster
      3. go to Addons section
      4. select Nvidia GPU card
      5. click on "uninstall" link

      Actual results:

      "redhat-nvidia-gpu-addon" is stuck under "terminating" status , so uninstall is not completed. 

      OCM rerports the addon as uninstalled (it seems there are no checks on the namespace status)

      Expected results:

      all the resources created by the operator must be deleted when uninstall is triggered

      Reproducibility (Always/Intermittent/Only Once):

      Reproduced 3/3, 2 different clusters

      Build Details:

      RHODS v1.13

      Nvidia GPU Addon 1.0.0

      Workaround:

      removing finalizers from the CRs which are blocking the uninstall

      Additional info:

              mresvani@redhat.com Michail Resvanis
              rhn-support-bdattoma Berto D'Attoma
              Berto D'Attoma Berto D'Attoma
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

                Created:
                Updated:
                Resolved: