Hello

We have an error with Debezium Connector for MySQL caused by an error on a bin log:

Caused by: io.debezium.DebeziumException: log event entry exceeded max_allowed_packet;
Increase max_allowed_packet on master; the first event 'mysql-bin.017548' at 64206627,
the last event read from '/var/lib/mysql/logbin/mysql-bin.017549' at 6716281,
the last byte read from '/var/lib/mysql/logbin/mysql-bin.017549' at 6716300.
Error code: 1236; SQLSTATE: HY000.

The current value for our parameters is:
max_allowed_packet is 32M
slave_max_allowed_packet is 1G
read_buffer_size is 262144

The event failing is bigger than the 1G (which is the maximum for the slave_max_allowed_packet ). We would like to know if there is any possibility on skipping the binlog that is failing. How would that be performed? Could there be any cut on the retrieved data?

Kind regards.

Bug report

For bug reports, provide this information, please:

What is the connector configuration?

  "config": {
    "connector.class": "io.debezium.connector.mysql.MySqlConnector",
    "snapshot.locking.mode": "none",
    "database.history.kafka.topic": "sbd_qas.history",
    "drop.deletes": "false",
    "include.schema.changes": "true",
    "decimal.handling.mode": "string",
    "_comment": "Kafka converter settings",
    "snapshot.new.tables": "parallel",
    "poll.interval.ms": "5000",
    "database.history.kafka.recovery.poll.interval.ms": "172800000",
    "database.history.skip.unparseable.ddl": "true",
    "value.converter": "io.confluent.connect.avro.AvroConverter",
    "key.converter": "io.confluent.connect.avro.AvroConverter",
    "database.user": "debezium",
    "database.server.id": "5545",
    "database.history.kafka.bootstrap.servers": xxxxxxx,
    "database.server.name": "xxxxxxxxxx",
    "database.port": "3306",
    "value.converter.schema.registry.url": xxxxxxxxxxxxxxxxxxxxx,
    "internal.key.converter": "org.apache.kafka.connect.json.JsonConverter",
    "database.serverTimezone": "Europe/Brussels",
    "database.hostname": "xxxxxxxxxxxxxxxxxxxxxxxxxx",
    "ddl.parser.mode": "antlr",
    "database.password": "Arinso01",
    "internal.value.converter": "org.apache.kafka.connect.json.JsonConverter",
    "name": "xxxxxxxxxxxxxxxxxxxxxxxxxx",
    "connect.keep.alive": "true",
    "key.converter.schema.registry.url": "xxxxxxxxxxxxxxxxxxxxxxxxx",
    "drop.tombstones": "false",
    "snapshot.mode": "when_needed"

What is the captured database version and mode of depoyment?

(E.g. on-premises, with a specific cloud provider, etc.)

Mysql enterprise 5.7.36

What behaviour do you see?

We have an error with Debezium Connector for MySQL caused by an error on a bin log:

Do you have the connector logs, ideally from start till finish?

(You might be asked later to provide DEBUG/TRACE level log)

"trace": "org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.\n\tat io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)\n\tat io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1217)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:980)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:599)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:857)\n\tat java.lang.Thread.run(Thread.java:750)\nCaused by: io.debezium.DebeziumException: log event entry exceeded max_allowed_packet; Increase max_allowed_packet on master; the first event 'mysql-bin.017548' at 24137554, the last event read from '/var/lib/mysql/logbin/mysql-bin.017549' at 6716281, the last byte read from '/var/lib/mysql/logbin/mysql-bin.017549' at 6716300. Error code: 1236; SQLSTATE: HY000.\n\tat io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1172)\n\t... 5 more\nCaused by: com.github.shyiko.mysql.binlog.network.ServerException: log event entry exceeded max_allowed_packet; Increase max_allowed_packet on master; the first event 'mysql-bin.017548' at 24137554, the last event read from '/var/lib/mysql/logbin/mysql-bin.017549' at 6716281, the last byte read from '/var/lib/mysql/logbin/mysql-bin.017549' at 6716300.\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:944)\n\t... 3 more\n"