Uploaded image for project: 'Cost Management'
  1. Cost Management
  2. COST-2809

Handle upcoming breaking changes related new optimize gcp downloader.

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Done
    • Icon: Critical Critical
    • 2022Q3
    • None
    • None

      The changes introduced in COST-2682 will be breaking changes in production due to:

      1. The new naming convention for our csv & parquet files in trino
      2. The new assembly id format we use in our manifest records

      In order to not break our GCP sources for our customers we will need to implement some self healing logic that does the following:

      1. Identifies gcp manifests related to our old downloader
      2. Removes the csv & parquet files for those manifests
      3. Deletes the manifests themselves from our database
      4. Resets the setup_complete flag for the provider
      5. Retriggers an initial download to recollect the gcp csv files and recreate the parquet files
      6. Can be run multiple times without affecting new manifests created by the new gcp downloader.

      This issue is to design and implement some some self healing code that will handle the above mentioned acceptance criteria.

            myersco Cody Myers
            myersco Cody Myers
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: