Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-64595

Remove Pending resources during destroy

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • 2
    • Important
    • None
    • None
    • Proposed
    • Installer Sprint 279
    • 1
    • In Progress
    • Bug Fix
    • Prior to the fix, a no-op error during the deprovision process could cause an endless attempt to delete a resource that does not exist. The fix removes pending resources marked for deletion when the resource cannot be found, concluding the endless loop.
    • None
    • None
    • None
    • None

      This is a clone of issue OCPBUGS-63711. The following is the description of the original issue:
      —
      Description of problem:

          When an error occurs during the deprovision process for GCP, the resulting action is to warn and skip over it (depending on the issue). This was found when we remove the firewalls.delete permission from the service account, and run the destroy process. The firewalls cannot be deleted, so we manually delete them. This caused the following issue 



      DEBUG Found firewall rule: bbarbach-gcp-test-n78nl-api 
DEBUG Found firewall rule: bbarbach-gcp-test-n78nl-control-plane 
DEBUG Found firewall rule: bbarbach-gcp-test-n78nl-etcd 
DEBUG Found firewall rule: bbarbach-gcp-test-n78nl-health-checks 
DEBUG Found firewall rule: bbarbach-gcp-test-n78nl-internal-cluster 
DEBUG Found firewall rule: bbarbach-gcp-test-n78nl-internal-network 
DEBUG Deleting firewall rule bbarbach-gcp-test-n78nl-internal-cluster 
DEBUG failed to delete firewall bbarbach-gcp-test-n78nl-internal-cluster: googleapi: Error 403: Required 'compute.firewalls.delete' permission for 'projects/openshift-dev-installer/global/firewalls/bbarbach-gcp-test-n78nl-internal-cluster', forbidden 
DEBUG Deleting firewall rule bbarbach-gcp-test-n78nl-internal-network 
DEBUG failed to delete firewall bbarbach-gcp-test-n78nl-internal-network: googleapi: Error 403: Required 'compute.firewalls.delete' permission for 'projects/openshift-dev-installer/global/firewalls/bbarbach-gcp-test-n78nl-internal-network', forbidden 
DEBUG Deleting firewall rule k8s-fw-a0f25490c0bef441c832c84ee93b41a5 
DEBUG failed to delete firewall k8s-fw-a0f25490c0bef441c832c84ee93b41a5: googleapi: Error 403: Required 'compute.firewalls.delete' permission for 'projects/openshift-dev-installer/global/firewalls/k8s-fw-a0f25490c0bef441c832c84ee93b41a5', forbidden 
DEBUG Deleting firewall rule k8s-a0f25490c0bef441c832c84ee93b41a5-http-hc 
DEBUG failed to delete firewall k8s-a0f25490c0bef441c832c84ee93b41a5-http-hc: googleapi: Error 403: Required 'compute.firewalls.delete' permission for 'projects/openshift-dev-installer/global/firewalls/k8s-a0f25490c0bef441c832c84ee93b41a5-http-hc', forbidden 
DEBUG Deleting firewall rule bbarbach-gcp-test-n78nl-api 
DEBUG failed to delete firewall bbarbach-gcp-test-n78nl-api: googleapi: Error 403: Required 'compute.firewalls.delete' permission for 'projects/openshift-dev-installer/global/firewalls/bbarbach-gcp-test-n78nl-api', forbidden 
DEBUG Deleting firewall rule bbarbach-gcp-test-n78nl-control-plane 
DEBUG failed to delete firewall bbarbach-gcp-test-n78nl-control-plane: googleapi: Error 403: Required 'compute.firewalls.delete' permission for 'projects/openshift-dev-installer/global/firewalls/bbarbach-gcp-test-n78nl-control-plane', forbidden 
DEBUG Deleting firewall rule bbarbach-gcp-test-n78nl-etcd 
DEBUG failed to delete firewall bbarbach-gcp-test-n78nl-etcd: googleapi: Error 403: Required 'compute.firewalls.delete' permission for 'projects/openshift-dev-installer/global/firewalls/bbarbach-gcp-test-n78nl-etcd', forbidden 
DEBUG Deleting firewall rule bbarbach-gcp-test-n78nl-health-checks 
DEBUG failed to delete firewall bbarbach-gcp-test-n78nl-health-checks: googleapi: Error 403: Required 'compute.firewalls.delete' permission for 'projects/openshift-dev-installer/global/firewalls/bbarbach-gcp-test-n78nl-health-checks', forbidden 
DEBUG Firewalls: 8 items pending

      We can see the 8 items pending. After we manually delete the firewalls the pending items are not causing a deletion error But they are still listed as pending. This will cause the destroy process to run indefinitely (unless restarted).

       

      Version-Release number of selected component (if applicable):

      How reproducible:

          Described Above

      Steps to Reproduce:

          1. Described above.
    2.
    3.

      Actual results:

         Destroy runs forever

      Expected results:

        Destroy completes

      Additional info:

              rh-ee-bbarbach Brent Barbachem
              rh-ee-bbarbach Brent Barbachem
              None
              None
              Jianli Wei Jianli Wei
              None
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: