Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7032

Kinesis Sink Reliability

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Done
    • Icon: Major Major
    • 2.5.0.Alpha2
    • None
    • debezium-server
    • None
    • False
    • None
    • False

      Feature request or enhancement

      For feature requests or enhancements, provide this information, please:

      Which use case/requirement will be addressed by the proposed feature?

      Everyone who uses the kinesis sink will be impacted by this new enhancement.

      Implementation ideas (optional)

      To make Kinesis sink more reliable and avoid the error 500, debezium should have a retry mechanism on handleBatch, waiting a success from the kinesis putRecord response.

      Besides that, it is interesting to change the putRecord method to putRecords, sending to kinesis a batch of records instead of one by one.

      https://repost.aws/knowledge-center/kinesis-data-stream-500-error

              Unassigned Unassigned
              brenoavm Breno Moreira (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: