-
Feature Request
-
Resolution: Done
-
Major
-
None
-
None
-
None
Further enhance/refine the built in materialization management to support alternative invalidation and refresh strategies.
In terms of priority, this would target external materialization first.
There must be an alternative to the implied default of the table rename strategy - which is problematic when a source is not transactional and should be changed to a merge/delete approach.
The general approaches are lazy (rows marked as dirty with refresh on query/schedule) and eager (which can even attempt to maintain close to transactional consistency). Optionally incremental loading can be used as well, to only load the portions of the materialization that are requested.
This requires more logic that we currently have:
- mapping source level row events to row level effects in materialized views (at worst we can refresh the whole row)
- additional status information possibly down to a column level, which much be added to the materialization table.
- incremental loading requires tracking the effectively covered predicate ranges and even columns of what has been loaded
It would probably be good to focus on a single scenario at first, such as:
external materialization, with non-incremental load (to reuse the existing simplistic load, but not rename logic), and eager refresh - and only considering transitive row effects based upon key relationships.
Then we can ensure that sufficient options are being added to account for additional scenarios.
- relates to
-
TEIID-4579 External Materialization MySQL can't swap primary and stage table in xa transaction
- Resolved
-
TEIIDDES-3028 Add "LoadNumber" column to Materialized View cache tables
- Resolved
-
TEIID-4738 Change JDG Materialization to use Upsert
- Resolved