Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7032

Kinesis Sink Reliability

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Done
    • Major
    • 2.5.0.Alpha2
    • None
    • debezium-server
    • None
    • False
    • None
    • False

    Description

      Feature request or enhancement

      For feature requests or enhancements, provide this information, please:

      Which use case/requirement will be addressed by the proposed feature?

      Everyone who uses the kinesis sink will be impacted by this new enhancement.

      Implementation ideas (optional)

      To make Kinesis sink more reliable and avoid the error 500, debezium should have a retry mechanism on handleBatch, waiting a success from the kinesis putRecord response.

      Besides that, it is interesting to change the putRecord method to putRecords, sending to kinesis a batch of records instead of one by one.

      https://repost.aws/knowledge-center/kinesis-data-stream-500-error

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              brenoavm Breno Moreira
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: