-
Bug
-
Resolution: Unresolved
-
Major
-
3.1.3.Final
-
None
-
False
-
-
False
-
Critical
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
For bug reports, provide this information, please:
What Debezium connector do you use and what version?
SQL Server Source Connect 3.1.3.Final
What is the connector configuration?
{
"task.max": 1,
"connector.class": "io.debezium.connector.sqlserver.SqlServerConnector",
"snapshot.mode":"no_data",
"incremental.snapshot.chunk.size": "30000",
"database.names": "test",
...
"table.include.list":"dbo.A,dbo.B,dbo.C,dbo.order,dbo.user, dbo.connector_signal",
"signal.enabled.channels": "source,kafka",
"signal.data.collection": "test.dbo.connector_signal",
"signal.kafka.topic": "signal-topic"
....
}
What is the captured database version and mode of deployment?
(E.g. on-premises, with a specific cloud provider, etc.)
- On-premises deployment
- Database version: SQL Server 2022
What behavior do you expect?
I expect the connector to resume incremental snapshot after restart
What behavior do you see?
I send a Kafka signal to trigger incremental snapshots for five tables. Initially, the connector was working perfectly and completed snapshotting first three tables.
While it was doing incremental snapshot for the fourth table, the connector was restarted. After restart, the connector stoped the incremental snapshots for the remaining tables. Checking the log, found "No incremental snapshot in progress, no action needed on start".
after restart, the offsets all look like this
{
"commit_lsn": "xxxx",
"snapshot": "INCREMENTAL",
"snapshot_completed": false
}
Before restart, the offsets looks like the following . Listed in the order as they occurred
{
"commit_lsn": "xxxx",
"snapshot": "INCREMENTAL",
"snapshot_completed": false
}
{
"incremental_snapshot_correlation_id": null,
"event_serial_no":2,
"incremental_snapshot_maximum_key": "xxxx",
"commit_lsn":"xxx":,
"change_lsn": :"xxx",
"incremental_snapshot_collections":"[{\"incremental_snapshot_collections_id\": \"test.dbo.order\",\"incremental_snapshot_collections_additional_condition\": null, \"incremental_snapshot_collections_surrogate_key\":null},{\"incremental_snapshot_collections_id\": \"test.dbo.user\",\"incremental_snapshot_collections_additional_condition\": null, \"incremental_snapshot_collections_surrogate_key\":null}]","incremental_snapshot_primary_key": "xxx"}
}
{
"commit_lsn": "xxxx",
"snapshot": "INCREMENTAL",
"snapshot_completed": false
}
{
"incremental_snapshot_correlation_id": null,
"event_serial_no":2,
"incremental_snapshot_maximum_key": "xxxx",
"commit_lsn":"xxx":,
"change_lsn": :"xxx",
"incremental_snapshot_collections":"[{\"incremental_snapshot_collections_id\": \"test.dbo.order\",\"incremental_snapshot_collections_additional_condition\": null, \"incremental_snapshot_collections_surrogate_key\":null},{\"incremental_snapshot_collections_id\": \"test.dbo.user\",\"incremental_snapshot_collections_additional_condition\": null, \"incremental_snapshot_collections_surrogate_key\":null}]","incremental_snapshot_primary_key": "xxx"}
}
{
"incremental_snapshot_correlation_id": null,
"event_serial_no":2,
"incremental_snapshot_maximum_key": "xxxx",
"commit_lsn":"xxx":,
"change_lsn": :"xxx",
"incremental_snapshot_collections":"[{\"incremental_snapshot_collections_id\": \"test.dbo.order\",\"incremental_snapshot_collections_additional_condition\": null, \"incremental_snapshot_collections_surrogate_key\":null},{\"incremental_snapshot_collections_id\": \"test.dbo.user\",\"incremental_snapshot_collections_additional_condition\": null, \"incremental_snapshot_collections_surrogate_key\":null}]","incremental_snapshot_primary_key": "xxx"}
}
{
"commit_lsn": "xxxx",
"snapshot": "INCREMENTAL",
"snapshot_completed": false
}
These are two formats of this connector offset I saw. The offset seem to alternate between these two formats
It seems this method determines what offset to commit.
When the incremental snapshot is in progress or has not finished yet, shouldn't the incremental snapshot info be always be part of the offset so it can resume after a restart ?
In my case, the snapshot can't be resumed because it can't find the collection of the tables to work on from the offset.
Do you see the same behaviour using the latest released Debezium version?
(Ideally, also verify with latest Alpha/Beta/CR version)
<Your answer>
Do you have the connector logs, ideally from start till finish?
(You might be asked later to provide DEBUG/TRACE level log)
<Your answer>
How to reproduce the issue using our tutorial deployment?
<Your answer>
Feature request or enhancement
For feature requests or enhancements, provide this information, please:
Which use case/requirement will be addressed by the proposed feature?
<Your answer>
Implementation ideas (optional)
<Your answer>
