Bug report

The KinesisChangeConsumer will send records in batches. If part of the batch fails it will try to resend these failed records. However, when attempting to resend failed records, the code uses indices incorrectly.

The problem is that putRecordsRequestEntryList is always the original full list of entries, but after the first retry, batchRequest becomes only the failed records. The response indices correspond to batchRequest entries, not the original list.

This mismatch causes:

Records that succeeded to be retried (duplicates)
Records that failed to never be retried (missing records)

What Debezium connector do you use and what version?

 Debezium 3.0.8. This bug was introduced when Kinesis batch putRecords was added.

What is the connector configuration?

N/A

What is the captured database version and mode of deployment?

N/A

What behavior do you expect?

KinesisChangeConsumer should retry records that failed in a batch

What behavior do you see?

KinesisChangeConsumer does not retry to records that failed, it sends the first N records in the original batch. N is the number of records that failed.

Do you see the same behaviour using the latest released Debezium version?

(Ideally, also verify with latest Alpha/Beta/CR version)

This bug was introduced in 3.0.1

Do you have the connector logs, ideally from start till finish?

Example log:
(pool-9-thread-1) Failed to send 100 number of records, retrying

How to reproduce the issue using our tutorial deployment?

<Your answer>

Feature request or enhancement

For feature requests or enhancements, provide this information, please:

Which use case/requirement will be addressed by the proposed feature?

<Your answer>

Implementation ideas (optional)

I have a working branch that fixes this Bug. I will create a pull request.