-
Enhancement
-
Resolution: Obsolete
-
Major
-
2.3.2.Final
-
False
-
-
False
What Debezium connector do you use and what version?
MySQL connector v2.3.2.Final
What is the connector configuration?
{
"name": "inventory-connector",
"config": {
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"tasks.max": "1",
"database.hostname": "mysql",
"database.port": "3306",
"database.user": "debezium",
"database.password": "dbz",
"database.server.id": "184054",
"snapshot.mode": "initial_only",
"topic.prefix": "dbserver1",
"database.include.list": "inventory",
"schema.history.internal.kafka.bootstrap.servers": "kafka:9092",
"signal.kafka.bootstrap.servers": "kafka:9092",
"schema.history.internal.kafka.topic": "schema-changes.inventory",
"signal.kafka.topic": "debezium_signal",
"signal.enabled.channels": "source,kafka",
"read.only": "true"
}
}
Change snapshot.mode to default (initial) to observe incremental snapshot does work with this setup
What is the captured database version and mode of depoyment?
docker deployment provided in the tutorial deployment: https://debezium.io/documentation/reference/2.3/tutorial.html
What behaviour do you expect?
Incremental snapshot should work with initial_only snapshot mode
What behaviour do you see?
Incremental request is successful but no actual incremental snapshot is done
Do you have the connector logs, ideally from start till finish?
Nothing related to incremental snapshot after the following logs, full log is attached as well logs.txt
:
2023-08-30 15:57:51,781 INFO || Requested 'INCREMENTAL' snapshot of data collections '[inventory.customers]' with additional condition 'No condition passed' and surrogate key 'PK of table will be used' [io.debezium.pipeline.signal.actions.snapshotting.ExecuteSnapshot]
2023-08-30 15:57:51,790 INFO || Incremental snapshot for table 'inventory.customers' will end at position [1004] [io.debezium.pipeline.source.snapshot.incremental.AbstractIncrementalSnapshotChangeEventSource]
How to reproduce the issue using our tutorial deployment?
Mostly follow the steps provided, some changes needed to reproduce include:
Enable GTID for read only incremental snapshot:
docker run -it --rm --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=debezium -e MYSQL_USER=mysqluser -e MYSQL_PASSWORD=mysqlpw quay.io/debezium/example-mysql:2.3 --gtid-mode=ON --enforce-gtid-consistency=ON
Trigger incremental snapshot with kafka-console-producer
kafka-console-producer.sh --topic debezium_signal --bootstrap-server host.docker.internal:9092 --property parse.key=true --property key.separator=":" >dbserver1:{"type":"execute-snapshot","data": {"data-collections": ["inventory.customers"], "type": "INCREMENTAL"}}