-
Epic
-
Resolution: Unresolved
-
Major
-
None
-
None
-
None
-
OCP on Cloud parquet processing split
-
35
-
False
-
None
-
False
-
To Do
-
36% To Do, 0% In Progress, 64% Done
OCP on Cloud parquet files are currently created during the download flow, this processing should be split into a separate celery task that runs Trino SQL. Large impact to the data processing pipeline, no impact to customer facing API/UI, update and document existing masu endpoint to use new flow.
This should be done one provider at a time in order to reduce risk.
This will be released one provider at a time to reduce risk of a larger change. AWS will be the first. Targeting stage by 2024/08/15.