-
Epic
-
Resolution: Done
-
Critical
-
None
-
None
-
None
-
Optimization GCP Downloader: manifests
-
False
-
None
-
False
-
0% To Do, 0% In Progress, 100% Done
GCP is different from our other providers since we build the csv files ourselves. Which should be handled differently in the day to day flow versus the initial downloaded. Therefore, the new implementation will check each day in the scan range to see if there is new data to be downloaded. We do this by collecting the partition date in GCP and mapping it to the last time data was exported to that partition:
SELECT DATE(_PARTITIONTIME), max(export_time) FROM `cloud-billing-292519.billing_datset.gcp_billing_export_resource_v1_018984_D0AAA4_940B88` WHERE DATE(_PARTITIONTIME) BETWEEN "2022-05-17" AND "2022-05-19" GROUP BY DATE(_PARTITIONTIME) – "2022-05-19"-2022-05-19 19:40:16.385000 UTC – Partitition_date:max(export_time)
Therefore, the new optimization will have a manifest per downloaded.
- is related to
-
COST-2761 [Case 03253656 ] Calculation Discrepancy with OpenShift on GCP for cluster and namespace amounts
- Closed