-
Feature Request
-
Resolution: Duplicate
-
Minor
-
None
-
0.9.0.Beta2
-
None
-
- Debezium 0.9.0Beta2
- wal2json commit d4c0e814696695bbf853c48b38b7479e0f83f6c7
- Postgresql 11.1
- confluent platform 5.1
- zookeeper 3.4.13
Current behavior
Debezium Postgresql connector does not support arbitrary-dimensional array. For example,
In case of two-dimensional array of INT
Postgresql connector warns it and the result value is null.
Avro schema
{
"name":"c_arr_int",
"type":[
"null",
{
"type":"array",
"items":[
"null",
"int"
],
"connect.parameters":{
"__debezium.source.column.type":"_INT4",
"__debezium.source.column.length":"10"
}
}
],
"default":null
}
the warning message
[stgresValueConverter] Unexpected value for JDBC type 4 and column c_arr_int-element int4(10, 0) DEFAULT VALUE NULL: class=Integer[]
In case of two-dimensional array of TEXT
Postgresql connector does not warn as follow and the result value seems to be casted string.
Avro schema
{
"name":"c_arr_text",
"type":[
"null",
{
"type":"array",
"items":[
"null",
"string"
],
"connect.parameters":{
"__debezium.source.column.type":"_TEXT",
"__debezium.source.column.length":"2147483647"
}
}
],
"default":null
}
result value
['[Ljava.lang.String;@3ce73358']
Request
- Warning should be logged.
At least, as long as Debezium does not support arbitrary-dimensional array, warning should be logged when it finds a two or more dimensional array of any data type. Moreover, if "include.unknown.datatypes" is enabled, raw binary representation of a column should be set as the result. - Support arbitrary-dimensional array
I guess the reason why Debezium does not support it is that both avro and protobuf can not define an arbitrary-dimensional array schema.
My suggestion is to define such array schema as a linear(flatten) array with the dimension infomation.