Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6775

Table schemas should be updated for each shard individually

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Done
    • Icon: Major Major
    • 2.4.0.Alpha2
    • None
    • vitess-connector
    • None

      Feature request or enhancement

      When Vitess schema is changed, it is applied to each shard individually. This means that the VStream will receive separate Query.Field events for each shard that represent that the schema has been changed for that shard. So, table schemas should be updated on a shard-by-shard basis, so that we can always correctly decode the messages received for that shard with the most up to date schema

      See this zulipchat for detailed discussion

      Which use case/requirement will be addressed by the proposed feature?

      See this zulipchat for detailed discussion, basically, users observe an error if a table schema is changed for a large enough keyspace where it is not applied instantly across all shards.

      Implementation ideas (optional)

      We need to encode the shard data with our table ID (only has schema/table) currently. This is the correct approach as each shard can have it's own distinct schema at a given time. It seems the best way to do this without massive refactor/reimplementation of core classes (e.g., Table, DatabaseSchema, DataCollectionID) is to simply insert the shard as the "catalog". We document this deviation from standard usage for the method that creates these augmented TableIDs.

              Unassigned Unassigned
              tthorn Thomas Thornton
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: