-
Bug
-
Resolution: Done
-
Major
-
0.7.0
-
None
debezium using wal2json returns different records from the initial sync and a subsequent update for the same record when the type is a custom/non-standard one (Postgis geometry in this case).
https://gist.github.com/rcoup/2ed1601577bb42d47db5f595f8d2ff46#file-update-json-L246 shows geom=null, whereas in the initial sync it has a (hex) value.
- initial.json is what you get for the row during --from-beginning
- update.json is what you get when you do UPDATE nz_titles SET id=id WHERE id=976948;
- log.txt is the associated kafka-connect/debezium log.
One thing that I noticed were some log messages about geom:
DEBUG Postgres|pgwal_test|records-stream-producer - field 'geom' (STRING) from column geom geometry(2147483647,0) [io.debezium.relational.TableSchemaBuilder] DEBUG Postgres|pgwal_test|records-stream-producer Mapped columns for table 'public.nz_titles' to schema: {"name" : "pgwal_test.public.nz_titles.Value", "type" : "STRUCT", "optional" : "true", "fields" : [..., {"name" : "geom", "index" : "11", "schema" : {"name" : "io.debezium.data.Json", "type" : "STRING", "optional" : "true", "version" : "1"}}]} [io.debezium.relational.TableSchemaBuilder]
The type is a geometry(MultiPolygon,2193) to be specific, so I'd imagine it'll just fall through into a string as it's unknown (as the first line implies). But the second line says io.debezium.data.Json. A chunk-o-hex won't validate as JSON, so maybe that's how it ends up null?