-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
None
-
False
-
None
-
False
-
Low
Bug report
What Debezium connector do you use and what version?
debezium-connector-ibmi-3.0.0.Alpha1
What is the connector configuration?
<Your answer>
What is the captured database version and mode of deployment?
(E.g. on-premises, with a specific cloud provider, etc.)
<Your answer>
What behavior do you expect?
updating a single row should result in one kafka event
What behavior do you see?
duplicate events for the same entry the second one missing the before image
Do you see the same behaviour using the latest released Debezium version?
yes 3.0.0.Alpha1
Do you have the connector logs, ideally from start till finish?
Looking at the logs:
```
2024-09-03 13:24:31,711 DEBUG [io.deb.con.db2.As400StreamingChangeEventSource] (debezium-as400rpcconnector-db2-change-event-source-coordinator) next event: 2024-09-03T13:24:38.543824Z - 543187362 type: UB table: TEST_TABLE
2024-09-03 13:24:31,711 DEBUG [io.deb.con.db2.As400StreamingChangeEventSource] (debezium-as400rpcconnector-db2-change-event-source-coordinator) next event: 2024-09-03T13:24:38.543824Z - 543187363 type: UP table: TEST_TABLE
2024-09-03 13:24:31,711 DEBUG [io.deb.con.db2.As400StreamingChangeEventSource] (debezium-as400rpcconnector-db2-change-event-source-coordinator) found before image for DEBEZIUM-TEST_TABLE
2024-09-03 13:24:31,711 DEBUG [io.deb.con.db2.As400StreamingChangeEventSource] (debezium-as400rpcconnector-db2-change-event-source-coordinator) update event id 543187363 tx 0 table B7031210.DEBEZIUM.TEST_TABLE
2024-09-03 13:24:45,208 DEBUG [io.deb.con.db2.As400StreamingChangeEventSource] (debezium-as400rpcconnector-db2-change-event-source-coordinator) next event: 2024-09-03T13:24:38.543824Z - 543187363 type: UP table: TEST_TABLE
2024-09-03 13:24:45,208 DEBUG [io.deb.con.db2.As400StreamingChangeEventSource] (debezium-as400rpcconnector-db2-change-event-source-coordinator) before image not found for DEBEZIUM-TEST_TABLE
```
it looks like we see the before image of the change offset 543187362
then the after image offset 543187363 and we tie the two events together: found before image for...
Then we find the after image again at the same offset 543187363 and this time we don't find the before image (which is correct as we deleted it the first time we found it)
I think this is a bug around marking the entry as processed. The API gives us a continuation offset - where to start next if there is more data or it's blank if there is no more data.
Our start position reflects this it's either set to the continuation offset or if we don't get one it's the last entry we processed.
If there is no continuation offset it should flag the last offset as processed so we skip it on the next call. I guess this isn't happening.
How to reproduce the issue using our tutorial deployment?
<Your answer>
Feature request or enhancement
For feature requests or enhancements, provide this information, please:
Which use case/requirement will be addressed by the proposed feature?
<Your answer>
Implementation ideas (optional)
almost certainly a bug in the `is processed` flag
I wonder if we should remove the handling of the continuation offset and always have the processed flag set to true
- is duplicated by
-
DBZ-8344 Some messages are duplicated (ibmi)
- Closed