Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3762

Exclude dummy events from database history


      I am using Debezium's mysql connector with MariaDB 10.5.10. MariaDB's binlog emits certain events that are unprocessable by Debezium. For example:

      {"source":{"server":"mysql-dbz-engine"},"position":{"transaction_id":null,"ts_sec":1626365177,"file":"binlog.000001","pos":471235611,"server_id":150,"event":1},"databaseName":"","ddl":"# Dummy event replacing event type 1","tableChanges":[]}
      {"source":{"server":"mysql-dbz-engine"},"position":{"transaction_id":null,"ts_sec":1626366177,"file":"binlog.000001","pos":471236611,"server_id":150,"event":160},"databaseName":"","ddl":"# Dummy event replacing event type 160","tableChanges":[]}

      Each of these events are persisted in Debezium's schema history.

      I tried a simple SMT filter, but it seems like it is downstream from whatever writes to the schema history. Since the DDL field is just a comment, can we exclude these from flooding the history topic? Or maybe provide an interface to apply SMT to the history topic itself?

            vjuranek@redhat.com Vojtech Juranek
            sgao103 Song Gao (Inactive)
            0 Vote for this issue
            4 Start watching this issue