-
Feature Request
-
Resolution: Duplicate
-
Minor
-
None
-
0.9.0.Beta2
-
None
Current behavior
Debezium Postgresql connector does not support arbitrary-dimensional array. For example,
In case of two-dimensional array of INT
Postgresql connector warns it and the result value is null.
Avro schema
{ "name":"c_arr_int", "type":[ "null", { "type":"array", "items":[ "null", "int" ], "connect.parameters":{ "__debezium.source.column.type":"_INT4", "__debezium.source.column.length":"10" } } ], "default":null }
the warning message
[stgresValueConverter] Unexpected value for JDBC type 4 and column c_arr_int-element int4(10, 0) DEFAULT VALUE NULL: class=Integer[]
In case of two-dimensional array of TEXT
Postgresql connector does not warn as follow and the result value seems to be casted string.
Avro schema
{ "name":"c_arr_text", "type":[ "null", { "type":"array", "items":[ "null", "string" ], "connect.parameters":{ "__debezium.source.column.type":"_TEXT", "__debezium.source.column.length":"2147483647" } } ], "default":null }
result value
['[Ljava.lang.String;@3ce73358']
Request
- Warning should be logged.
At least, as long as Debezium does not support arbitrary-dimensional array, warning should be logged when it finds a two or more dimensional array of any data type. Moreover, if "include.unknown.datatypes" is enabled, raw binary representation of a column should be set as the result. - Support arbitrary-dimensional array
I guess the reason why Debezium does not support it is that both avro and protobuf can not define an arbitrary-dimensional array schema.
My suggestion is to define such array schema as a linear(flatten) array with the dimension infomation.