Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7089

Enhance Debezium Server: Implement PutRecords for Kinesis Sink Integration

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False

      Feature request or enhancement

      For feature requests or enhancements, provide this information, please:

      Which use case/requirement will be addressed by the proposed feature?

      The current configuration of the Debezium Server for the Kinesis Sink employs the PutRecord method for data transmission. However, based on information from AWS documentation, it appears that the PutRecords method offers superior performance characteristics. Consequently, it is advisable to investigate and implement the PutRecords method if this assessment is accurate.

      Implementation ideas (optional)

      To potentially enhance the Debezium Server's performance when working with the Kinesis Sink, we should consider transitioning from the PutRecord function to the PutRecords function. This transition can be carried out after a thorough examination of the AWS documentation provided here to verify the claimed performance improvements.

       

      Here are their statement:

      There are two different operations in the Kinesis Data Streams API that add data to a stream, PutRecords and PutRecord. The PutRecords operation sends multiple records to your stream per HTTP request, and the singular PutRecord operation sends records to your stream one at a time (a separate HTTP request is required for each record). You should prefer using PutRecords for most applications because it will achieve higher throughput per data producer. 

              Unassigned Unassigned
              meilyasahsan ilyas ahsan (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: