-
Sub-task
-
Resolution: Done
-
Normal
-
None
-
None
-
None
-
1
-
COST-10 - New data architecture that includes Data Hub as big data pipeline
-
COST Sprint 50, COST Sprint 51, COST Sprint 52, COST Sprint 53
We used OpenShift as the prototype for table partitioning. The linked PR (https://github.com/project-koku/koku/pull/2256) should serve as the template for other source types.
The openshift daily summary tables as well as the aws and azure summary tables will now be partitioned by year and month.
Testing
New DB
- git checkout master && git pull
- make docker-reinitdb-with-sources
- psql postgresql://postgres@localhost:15432/postgres
- \dt acct10001.*
- \q
You should see three daily tables listed as partitioned table followed by another table named <daily_table>_default.
Existing DB With Data
- git checkout bbb3253aa3ff0d727ae13175afa6b5b311ba904b && git pull
- make docker-down && make docker-up && make docker-reinitdb-with-sources
- Create some data for ocp, azure, aws
- git checkout master && git pull
- make run-migrations
- psql postgresql://postgres@localhost:15432/postgres
- \dt acct10001.*
- \q
You should see the partitioned tables as well as some similarly named tables ending with YYYY_MM to designate the partition. There should also be a default partition.
None of the default partitions should have any data
- blocks
-
COST-413 Create new partitioned tables via migration
- Closed