Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5410

Enhancing Cassandra 4 Connector to read incremental changes and not wait for Commit Log file to be marked complete

XMLWordPrintable

      Cassandra Log Reader provides API to read the CommitLog file upto to the listed offset in the _cdc.idx file making near real time streaming possible. 

      https://cassandra.apache.org/doc/latest/cassandra/operating/cdc.html
      The same principle in mentioned in Debezium Documentation
       
      "In Cassandra 4.0 on every segment fsync, an index file will be updated to reflect latest offset. This will eliminate the processing delay in the CDC feature in Cassandra 3.X. This connector feature is to be added in the future with the Cassandra 4.0 release."
      https://debezium.io/documentation/reference/stable/connectors/cassandra.html
       
      We have made this enhancement and running with this feature since last few weeks under high load scenarios would love to contribute back via a PR to Debezium code base and this Jira serves as starting point. 

      + smiklosovic jpechane  

       

            Unassigned Unassigned
            nitin.nitt Nitin Chhabra (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: