-
Bug
-
Resolution: Done
-
Major
-
None
-
None
-
False
-
None
-
False
Bug report
What Debezium connector do you use and what version?
2.7.1.Final
What is the connector configuration?
{ "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "database.hostname": "postgres", "database.port": "5432", "database.user": "postgres", "database.password": "postgres", "database.dbname": "test_database", "database.server.name": "test", "plugin.name": "pgoutput", "table.include.list": "test_table", "signal.data.collection": "public.debezium_signals", "slot.name": "debezium_test", "post.processors": "reselector", "reselector.type": "io.debezium.processors.reselect.ReselectColumnsPostProcessor", "reselect.columns.include.list": "public.test_table:jsonb_field,public.test_table:text_field", "reselect.unavailable.values": "true", "reselect.null.values": "false", "reselect.use.event.key": "false", "topic.prefix": "prefix" }
What is the captured database version and mode of deployment?
Postgres 11-3.1 on docker
What behavior do you expect?
JSONB data to be reselected, instead of having "__debezium_unavailable_value" as a value
What behavior do you see?
TOASTed JSONB never reselected, with or without a default value set, also with or without NOT NULL constraint .. always shows "__debezium_unavailable_value"
Do you see the same behaviour using the latest released Debezium version?
Issue exists on both 2.7.1.FINAL and 3.0.0.Alpha2
Do you have the connector logs, ideally from start till finish?
2024-08-21 15:32:31,741 DEBUG Postgres|prefix|streaming Relation '124334' resolved to table 'public.test_table' [io.debezium.connector.postgresql.PostgresSchema] 2024-08-21 15:32:31,742 DEBUG Postgres|prefix|streaming Adding column text_field for table public.test_table to re-select list due to unavailable value placeholder. [io.debezium.processors.reselect.ReselectColumnsPostProcessor] 2024-08-21 15:32:32,142 DEBUG || Sending 1 records to topic prefix.public.test_table [io.debezium.connector.common.BaseSourceTask] 2024-08-21 15:32:32,142 INFO || 1 records sent during previous 00:03:17.9, last recorded offset of {server=prefix} partition is {lsn_proc=618714264, messageType=UPDATE, lsn_commit=618713600, lsn=618714264, txId=63695, ts_usec=1724254351326042} [io.debezium.connector.common.BaseSourceTask]
How to reproduce the issue using our tutorial deployment?
- Create a postrges table with the following schema:
CREATE TABLE public.test_table ( id serial4 NOT NULL, jsonb_field jsonb DEFAULT '{}'::jsonb NOT NULL, text_field text NULL, dummy_field text null, CONSTRAINT test_table_pkey PRIMARY KEY (id) );
- Insert a record with more than 4kb data in the json_field/text_field, SAMPLE:
https://pastebin.com/fMckNXvp
- Add new connector with the previously mentioned configuration
- Update the record and monitor the value of the fields
update test_table set dummy_field = '123' where id = 1
- Full value from kafka
https://pastebin.com/5mknHL68
- links to
-
RHEA-2024:139598 Red Hat build of Debezium 2.5.4 release