Details
-
Enhancement
-
Resolution: Won't Do
-
Major
-
None
-
1.7.0.Alpha1
-
None
-
False
-
False
-
Undefined
Description
Some source connectors (Debezium Cassandra connector) generate CDC records with struct-wrapped values. For example, when a table has an integer "id" and string "name" fields, CDC records from Postgre or MySQL may contain something like
"after" : { "id" : 2, "name" : "George Baily" }
whereas CDC records from Cassandra connector may contain struct-wrapped values that look like
"after" : { "id" : { "deletion_ts" : null, "set" : true, "value" : 2 }, "name" : { "deletion_ts" : null, "set" : true, "value" : "George Baily" } }
As a result, applications that work for Debezium MySQL or Postgre connectors cannot be directly used with Cassandra connector.
One approach to solve this problem would be to add an option in Cassandra connector to generate the plain records like the other connectors, however this approach was not favoured by the Cassandra connector maintainer. Therefore, I would propose to add an SMT to transform Cassandra's struct-wrapped CDC records to the plain CDC records. This approach is non-invasive and allows the sink connectors to transparently work with both types of records.
The original posting google group https://groups.google.com/g/debezium/c/XKa8v0ovkis/m/9Nnxg8V1AgAJ