Details

      Description

      AFAIK right now there are these configuration options to change how exceptions are handled:
      *  event.processing​.failure.handling.mode
      inconsistent.schema.handling.mode

      Also could be considered related 
      database.history.skip.unparseable.ddl

      Kafka Connect side handlers:
      "errors.tolerance": "all",
      "errors.log.enable":true,
      "errors.log.include.messages":true,

      + any others?

      Yet it seems it's not enough.

      For example, when the Debezium internal database schema and MySQL don't match up, then that exception can't be handled gracefully.

      org.apache.kafka.connect.errors.ConnectException: The binlog event does not contain expected number of columns; the internal schema representation is probably out of sync with the real database schema, or the binlog contains events recorded with binlog_row_image other than FULL or the table in question is an NDB table

      To reproduce just alter some table with session sql_log_bin=0 and then insert data into that table.

      So there are two features that would help operations:

      • Catch all configurable exception handler - to tell I'm OK with data loss for only that problematic table, but please keep working on the rest of tables/databases.
      • Ability to log out what table/database actually caused that issue.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                pimpelsang Eero Koplimets
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: