Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3760

Debezium snapshot.select.statement.overrides overrides not used

XMLWordPrintable

    • False
    • False
    • Undefined
    • Hide

      configure debezium server like below and run snapshot step

      use snapshot override for a table(or database) which has `_` in name

      config.put("%mysql.debezium.source.database.include.list", "inventory");
 config.put("debezium.source.table.include.list", "inventory.customers,inventory.orders,inventory.products,inventory.products_on_hand");
 config.put("debezium.source.snapshot.select.statement.overrides.inventory.products_on_hand", "SELECT * FROM products_on_hand WHERE 1>2");
      Show
      configure debezium server like below and run snapshot step use snapshot override for a table(or database) which has `_` in name config.put( "%mysql.debezium.source.database.include.list" , "inventory" ); config.put( "debezium.source.table.include.list" , "inventory.customers,inventory.orders,inventory.products,inventory.products_on_hand" ); config.put( "debezium.source.snapshot.select.statement.overrides.inventory.products_on_hand" , "SELECT * FROM products_on_hand WHERE 1>2" );

      seems like with debezium 1.6.Final  `snapshot.select.statement.overrides` is not picked up correctly

      `_`. character in the table names replaced by `.` dot which might be source of  the issue

      adding below Mysql unit-test log which shows the issue

       

       2021-07-15 09:23:25,213 INFO [io.deb.con.com.BaseSourceTask] (pool-11-thread-1) database.server.name = testc
>>>> 2021-07-15 09:23:25,213 INFO [io.deb.con.com.BaseSourceTask] (pool-11-thread-1) snapshot.select.statement.overrides.inventory.products.on.hand = SELECT * FROM products_on_hand WHERE 1>2
 2021-07-15 09:23:25,214 INFO [io.deb.con.com.BaseSourceTask] (pool-11-thread-1) transforms.unwrap.add.fields = op,table,source.ts_ms,db
 2021-07-15 09:23:25,214 INFO [io.deb.con.com.BaseSourceTask] (pool-11-thread-1) table.include.list = inventory.customers,inventory.orders,inventory.products,inventory.test_date_table,inventory.products_on_hand,inventory.geom,inventory.table_datatypes
 2021-07-15 09:23:25,214 INFO [io.deb.con.com.BaseSourceTask] (pool-11-thread-1) database.history = io.debezium.relational.history.FileDatabaseHistory
 2021-07-15 09:23:25,215 INFO [io.deb.con.com.BaseSourceTask] (pool-11-thread-1) database.include.list = inventory

..........

2021-07-15 09:23:26,772 INFO [io.deb.con.mys.MySqlSnapshotChangeEventSource] (debezium-mysqlconnector-testc-change-event-source-coordinator) Reading structure of database 'inventory'
 2021-07-15 09:23:26,901 INFO [io.deb.rel.RelationalSnapshotChangeEventSource] (debezium-mysqlconnector-testc-change-event-source-coordinator) Snapshot step 6 - Persisting schema history
 2021-07-15 09:23:26,970 INFO [io.deb.con.mys.MySqlSnapshotChangeEventSource] (debezium-mysqlconnector-testc-change-event-source-coordinator) Releasing global read lock to enable MySQL writes
 2021-07-15 09:23:26,982 INFO [io.deb.con.mys.MySqlSnapshotChangeEventSource] (debezium-mysqlconnector-testc-change-event-source-coordinator) Writes to MySQL tables prevented for a total of 00:00:01.132
 2021-07-15 09:23:26,982 INFO [io.deb.rel.RelationalSnapshotChangeEventSource] (debezium-mysqlconnector-testc-change-event-source-coordinator) Snapshot step 7 - Snapshotting data
 2021-07-15 09:23:26,983 INFO [io.deb.rel.RelationalSnapshotChangeEventSource] (debezium-mysqlconnector-testc-change-event-source-coordinator) Snapshotting contents of 5 tables while still in transaction
........

2021-07-15 09:23:27,073 INFO [io.deb.rel.RelationalSnapshotChangeEventSource] (debezium-mysqlconnector-testc-change-event-source-coordinator) Exporting data from table 'inventory.products_on_hand' (4 of 5 tables)

>>>> 2021-07-15 09:23:27,073 INFO [io.deb.rel.RelationalSnapshotChangeEventSource] (debezium-mysqlconnector-testc-change-event-source-coordinator) For table 'inventory.products_on_hand' using select statement: 'SELECT * FROM `inventory`.`products_on_hand`'
 2021-07-15 09:23:27,093 INFO [io.deb.rel.RelationalSnapshotChangeEventSource] (debezium-mysqlconnector-testc-change-event-source-coordinator) Finished exporting 9 records for table 'inventory.products_on_hand'; total duration '00:00:00.021'

       

       

      this lines shows the issue

      >> 2021-07-15 09:23:25,213 INFO [io.deb.con.com.BaseSourceTask] (pool-11-thread-1) snapshot.select.statement.overrides.inventory.products.on.hand = SELECT * FROM products_on_hand WHERE 1>2 

      select is not overriden

      >> 2021-07-15 09:23:27,073 INFO [io.deb.rel.RelationalSnapshotChangeEventSource] (debezium-mysqlconnector-testc-change-event-source-coordinator) For table 'inventory.products_on_hand' using select statement: 'SELECT * FROM `inventory`.`products_on_hand`'

       cc: gunnar.morling

              anmohant Anisha Mohanty
              ismailsimsek ismail simsek (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: