-
Task
-
Resolution: Unresolved
-
Normal
-
None
-
None
- Review existing indexes on the tally_snapshots table.
- Create a new partial index that can help with read and doesn't affect write process or update existing indexes.
- Check if runInTransaction=false helps since CREATE INDEX CONCURRENTLY cannot run inside a transaction and having this avoids locking issues on the 198M row table. https://docs.liquibase.com/reference-guide/changelog-attributes/runintransaction
- Ensure the index aligns with current query patterns and filtering conditions. Not sure if adding snapshot_date in index will be helpful.
- Include migration and rollback support for index changes.
- Since is_primary defaults to false and index is WHERE true, the index starts empty (should be fast even with 198M)
- Determine a way to add the index concurrently (to avoid locking the table) and without creating an outage (which a Liquibase run would do). See the comments for more information on how to do this.
Example:
<changeSet id="" author="" runInTransaction="false"> <sql> CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tally_snapshots_primary ON tally_snapshots (org_id, product_id, granularity) WHERE is_primary = true; </sql> <rollback> DROP INDEX IF EXISTS idx_tally_snapshots_primary; </rollback> </changeSet>
We can also try the same locally, by directly doing below for testing purpose
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tally_snapshots_primary
ON tally_snapshots (org_id, product_id, granularity)
WHERE is_primary_record = true;
AC:
- The index will be added to the table in production manually through an out of band procedure to avoid Liquibase related issues. After completion in production we can create a PR to apply the same index to the lower environments.
- No functional impact to existing queries or data.
- No execution time degradation when running queries involving tally_snapshots (can be poked by automated component tests and compared against historical execution time of the same tests from report portal)
- Download prod data locally for tally_snapshots and tally_measurements and make sure the index gets used and time it takes to apply index. QE and DEV need to do the same.
- is blocked by
-
SWATCH-4491 Add is_primary column to tally_snapshots Table and update JPA Entity
-
- Release Pending
-
- mentioned on
(1 mentioned on)