Uploaded image for project: 'Cost Management'
  1. Cost Management
  2. COST-2809

Handle upcoming breaking changes related new optimize gcp downloader.

    XMLWordPrintable

Details

    • Task
    • Resolution: Done
    • Critical
    • 2022Q3
    • None
    • None

    Description

      The changes introduced in COST-2682 will be breaking changes in production due to:

      1. The new naming convention for our csv & parquet files in trino
      2. The new assembly id format we use in our manifest records

      In order to not break our GCP sources for our customers we will need to implement some self healing logic that does the following:

      1. Identifies gcp manifests related to our old downloader
      2. Removes the csv & parquet files for those manifests
      3. Deletes the manifests themselves from our database
      4. Resets the setup_complete flag for the provider
      5. Retriggers an initial download to recollect the gcp csv files and recreate the parquet files
      6. Can be run multiple times without affecting new manifests created by the new gcp downloader.

      This issue is to design and implement some some self healing code that will handle the above mentioned acceptance criteria.

      Attachments

        Issue Links

          Activity

            People

              myersco Cody Myers
              myersco Cody Myers
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: