Details
-
Enhancement
-
Resolution: Unresolved
-
Major
-
1.2.0.Final
-
None
Description
AFAIK right now there are these configuration options to change how exceptions are handled:
* event.processing.failure.handling.mode
* inconsistent.schema.handling.mode
Also could be considered related
* database.history.skip.unparseable.ddl
Kafka Connect side handlers:
"errors.tolerance": "all",
"errors.log.enable":true,
"errors.log.include.messages":true,
+ any others?
Yet it seems it's not enough.
For example, when the Debezium internal database schema and MySQL don't match up, then that exception can't be handled gracefully.
org.apache.kafka.connect.errors.ConnectException: The binlog event does not contain expected number of columns; the internal schema representation is probably out of sync with the real database schema, or the binlog contains events recorded with binlog_row_image other than FULL or the table in question is an NDB table
To reproduce just alter some table with session sql_log_bin=0 and then insert data into that table.
So there are two features that would help operations:
- Catch all configurable exception handler - to tell I'm OK with data loss for only that problematic table, but please keep working on the rest of tables/databases.
- Ability to log out what table/database actually caused that issue.