Uploaded image for project: 'OpenShift Installer'
  1. OpenShift Installer
  2. CORS-1599

AWS deprovision: periodically re-terminate shutting-down-for-a-while instances

    XMLWordPrintable

Details

    • Story
    • Resolution: Done
    • Undefined
    • None
    • None
    • None
    • None
    • 3
    • False
    • False
    • Undefined
    • Sprint 200, Sprint 201

    Description

      We had a dozen or so instances in the CI account that were stuck in shutting-down for days. eparis@redhat.com opened an AWS ticket, but they didn't really investigate. Requesting a fresh termination removed the instances. Word from AWS folks is that being stuck for 15m or more is a sign of trouble, so logging a warning and re-terminating any instances at least that often seems reasonable.

      Sometimes re-terminating helps, and sometimes it doesn't (or maybe that didn't quite get as far as re-terminating?), but asking for a fresh termination every 15m or so doesn't seem like it would have negative consequences. Log line should definitely whine about AWS not terminating ("consider filing a ticket with AWS support").

      Attachments

        Issue Links

          Activity

            People

              rna-afk Aditya Narayanaswamy
              trking W. Trevor King
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: