Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6219

Highlight information about how to configure the schema history topic to store data only for intended tables

XMLWordPrintable

      Discussed on Zulip, 17 Feb, 2023

      Customers frequently raise questions about how to configure the history topic so that it reads information about only the tables that they designate for capture. 

      To provide users with a better understanding of how to configure the schema history so that connectors capture the schema history only for tables that are captured, the documentation should provide explicit guidance, with an example.
      Although the current documentation includes information in a note, to better highlight the information, it should be presented under a task-based heading.
      For example, Configuring the connector to capture the schema history only for tables that it captures data from.

      What's the best way to configure the schema history so that it captures the schema history only for tables that are captured?

      • Add the explicit schema. table name to the exclude list?
      • Set schema.history.internal.store.only.captured.tables.ddl to false?

      The schema.exclude.list is one way to define whether the connector should capture data for a specific schema, just like table.exclude.list is.
      However, due to the mutual exclusivity of the include and exclude filters, this can be problematic in some situations, depending on the connector, where the table resides, and the current connector configuration.

      Configuring schema history capture is different than configuring the change events that you want a connector to capture.

      Depending on the setting of store.only.captured.tables.ddl property, the connector captures the schema history for one of the following sets of tables:

      • When the property is set to false, the connector captures the schema for all tables in the database we're connected to, except for any built-in schemas that are hardcoded as exclusions.
      • When the property is set to false, the connector captures the schema only of the tables that match the include/exclude filter mappings.

      Applies to all connectors except PostgreSQL and MongoDB, which do not have schema history topics.

            broldan@redhat.com Robert Roldan
            broldan@redhat.com Robert Roldan
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: