Uploaded image for project: 'Cost Management'
  1. Cost Management
  2. COST-5129

Create OCP on Cloud parquet processing manged trino tables

XMLWordPrintable

    • Icon: Epic Epic
    • Resolution: Done
    • Icon: Major Major
    • 2024-Nov-08
    • None
    • None
    • OCP on Cloud parquet processing split
    • 23
    • False
    • None
    • False
    • To Do
    • 0% To Do, 0% In Progress, 100% Done

      OCP on Cloud parquet files are currently created during the download flow, this processing should be split into a separate celery task that runs Trino SQL. Large impact to the data processing pipeline, no impact to customer facing API/UI, update and document existing masu endpoint to use new flow.

       

      This should be done one provider at a time in order to reduce risk.

      Design Doc

       

      This will be released one provider at a time to reduce risk of a larger change. AWS will be the first.  Targeting stage by 2024/08/15.

       

              myersco Cody Myers
              cgoodfre Corey Goodfred (Inactive)
              Daniel Chorvatovic Daniel Chorvatovic
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

                Created:
                Updated:
                Resolved: