Major Raise ticket based on recommendation from
Chris Cranford (Naros)(zulip chat link):
Can you raise a Jira for this @{}Ivan Tsang{} so we can maybe make this configurable. It looks like the behavior is currently hardcoded to return null in this case:
case ARRAY: ArrayNode arrayNode = (ArrayNode) node; return arrayNode.isEmpty() ? null : SchemaBuilder.array(toConnectSchemaWithCycles(key, arrayNode)).optional().build();
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
For bug reports, provide this information, please:
What Debezium connector do you use and what version?
Debezium Postgres Connector, 2.4.0 Final
What is the connector configuration?
{
"name": "dbz-outbox",
"config" : {
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"plugin.name": "pgoutput",
"database.hostname": "'"$DB_HOST"'",
"database.port": "'"$DB_PORT"'",
"database.user": "'"$DB_USER"'",
"database.password": "'"$DB_PASSWORD"'",
"database.dbname": "'"$DB_NAME"'",
"database.server.name": "'"OUTBOX_$DB_NAME"'",
"topic.prefix": "notused",
"transforms": "outbox",
"transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter",
"transforms.outbox.route.by.field": "topic",
"transforms.outbox.route.topic.replacement": "${routedByValue}",
"transforms.outbox.table.field.event.key": "key",
"transforms.outbox.table.expand.json.payload": "true",
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"key.converter.schemas.enable": "false",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"value.converter.schemas.enable": "false",
"transforms.outbox.table.json.payload.null.behavior": "optional_bytes",
"tasks.max": "1",
"slot.drop.on.stop": "false",
"slot.name" : "'"$OUTBOX_REPLICATION_SLOT_NAME"'",
"publication.name" : "'"$OUTBOX_PUBLICATION_NAME"'",
"publication.autocreate.mode" : "disabled",
"snapshot.mode" : "never"
}
}
What is the captured database version and mode of deployment?
Running local docker-compose with postgres:14.3
What behaviour do you expect?
After inserting this into the value into the `payload` column in my outbox table
{"arrayKey1": [], "key1": "value1", "key2": "value2"}
I expect to see this in my kafka topic message
{
"arrayKey1": [
],
"key1": "value1",
"key2": "value2"
}
What behaviour do you see?
I see this incorrect kafka message
{
"key1": "value1",
"key2": "value2"
}
Do you see the same behaviour using the latest released Debezium version?
yes, I used debezium/connect:2.4.0.Final
Do you have the connector logs, ideally from start till finish?
(You might be asked later to provide DEBUG/TRACE level log)
If needed I can produce.
How to reproduce the issue using our tutorial deployment?
N/A