-
Bug
-
Resolution: Unresolved
-
Critical
-
None
-
False
-
-
False
Hello.
I'm using Debezium with MariaDB. I have snapshot mode set to 'when_needed'.
I got the same issue when using 3.0.6.Final and 3.2.0.Final
I have enabled binlog on a mariadb host with several databases on it. I have source connectors attached to some of them. In some of those dbs, it is written not as often as the others. For example one of the dbs might not see new events for more than 2 weeks, while in the others is written on a daily basis. The binlog retention is 1 week.
I have set the snapshot.mode to when_needed on the connecctor for the db that is rarely written to, assuming that a snapshot will be triggered if a binlog/gtid is lost.
However, upon restart i get the following error:
io.debezium.DebeziumException: Could not find GTID state requested by slave in any binlog files. Probably the slave state is too old and required binlog files have been purged. Error code: 1236; SQLSTATE: HY000. at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.wrap(BinlogStreamingChangeEventSource.java:1206) at io.debezium.connector.binlog.BinlogStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(BinlogStreamingChangeEventSource.java:1252) at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1110) at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:657) at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:959) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.github.shyiko.mysql.binlog.network.ServerException: Could not find GTID state requested by slave in any binlog files. Probably the slave state is too old and required binlog files have been purged. at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1074) ... 3 common frames omitted
{{}}
The snapshot wasn't triggered.
Bug report
For bug reports, provide this information, please:
What Debezium connector do you use and what version?
3.0.6.Final and 3.2.0.Final
What is the connector configuration?
connector.class = io.debezium.connector.mariadb.MariaDbConnector bootstrap.servers = ******* signal.enabled.channels = kafka include.schema.changes = true connector.adapter = mariadb database.jdbc.driver = org.mariadb.jdbc.Driver record.processing.threads = offset.storage.topic = ******* database.history.store.only.captured.tables.ddl = false errors.retry.delay.initial.ms = 300 key.converter = org.apache.kafka.connect.json.JsonConverter metrics.jmx.enabled = true database.user = ******* signal.kafka.bootstrap.servers = ******* offset.storage = org.apache.kafka.connect.storage.KafkaOffsetBackingStore signal.kafka.topic = ******* schema.history.internal.kafka.bootstrap.servers = ******* internal.task.management.timeout.ms = 8000000 database.protocol = jdbc:mariadb errors.max.retries = -1 database.password = ******** name = ******* connect.keep.alive = true snapshot.mode = when_needed database.timezone = disable record.processing.shutdown.timeout.ms = 1000 database.history.producer.max.request.size = 33554432 schema.history.internal.store.only.captured.databases.ddl = true record.processing.order = ORDERED topic.prefix = ******* offset.storage.file.filename = schema.history.internal.kafka.topic = ******* offset.storage.partitions = 1 database.history.skip.unparseable.ddl = true connections.max.idle.ms = 540000 value.converter = org.apache.kafka.connect.json.JsonConverter database.history.producer.buffer.memory = 33554432 database.server.id = 9911 offset.flush.timeout.ms = 10000 notification.enabled.channels = sink database.server.name = ******* errors.retry.delay.max.ms = 10000 snapshot.max.threads = 1 database.history.consumer.max.partition.fetch.bytes = 33554432 database.port = 53260 notification.sink.topic.name = ******* offset.flush.interval.ms = 20000 database.ssl.mode = disable record.processing.with.serial.consumer = false database.hostname = ******* offset.storage.replication.factor = 2 database.include.list = *******
What is the captured database version and mode of deployment?
(E.g. on-premises, with a specific cloud provider, etc.)
Mariadb AWS RDS
What behavior do you expect?
An initial snapshot to be triggered when the snapshot mode is set to when_needed and a missign gtid is found.
What behavior do you see?
It throws the above mention error and doesn't initiate a snapshot.
Do you see the same behaviour using the latest released Debezium version?
yes
Do you have the connector logs, ideally from start till finish?
How to reproduce the issue using our tutorial deployment?
Zulip refference link: https://debezium.zulipchat.com/#narrow/channel/348104-community-mysql-mariadb/topic/Snapshot.20mode.20when_needed.20issue