Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1405

Replace Custom Schema with Pluggable Serializers via KC Schema in Cassandra Connector

    Details

      Description

      The idea is to make the aspect of serialization configurable, i.e. how the events created by the connector are converted into the binary representation sent to Kafka. Kafka Connect has the notion of converters for this which take a KC Record and convert to/from binary (see the "key.converter" and "value.converter" of KC). Currently, as the Cassandra connector is not based on KC, it's exclusively using Avro (via GenericRecord). Ideally, the existing Converter contract from KC (and thus all the existing implementations such as Avro and JSON or custom ones) would be supported. For that, it'd have to change how messages are produced: instead of assembling Avro's GenericRecord, this would have to be done using KC's SourceRecord. That'd establish add a dependency to the KC API which might seem surprising at first, but I think it's advantageous eventually, as it allows for re-use of all the existing converters and makes for a consistent configuration of this matter with the other, actually KC-based connectors.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                bingqinzhou Bingqin Zhou
                Reporter:
                jgao54 Joy Gao
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: