In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

Bug report

For bug reports, provide this information, please:

What Debezium connector do you use and what version?

I am using the Postgres connector version 2.2

What is the connector configuration?

Running this with Debezium server standalone,

 

`application.properties`
debezium.source.snapshot.mode=initial
debezium.source.connector.class=io.debezium.connector.postgresql.PostgresConnector
debezium.source.database.hostname=postgres
debezium.source.database.port=5432
debezium.source.database.user=postgres
debezium.source.database.password=postgres
debezium.source.database.dbname=postgres
debezium.source.topic.prefix=dbserver1
debezium.source.plugin.name=pgoutput

What is the captured database version and mode of depoyment?

Postgres 13 on AWS RDS

What behaviour do you expect?

Return the column `activity_items = __debezium_unavailable_value` if the column value is TOASTed

What behaviour do you see?

It crashes with an exception (please see connector logs below).

 

Do you see the same behaviour using the latest relesead Debezium version?

Yes.

Do you have the connector logs, ideally from start till finish?

(You might be asked later to provide DEBUG/TRACE level log)

{"timestamp":"2023-06-07T18:56:41.686Z","sequence":229,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.relational.TableSchemaBuilder","level":"ERROR","message":"Failed to properly convert data value for 'public.community.activity_items' of type _uuid","threadName":"debezium-postgresconnector-0feec078-acf3-47bb-a5a3-b999b8842032.b769ab5a-e00a-4e34-916e-b4f35d19ebb0-change-event-source-coordinator","threadId":31,"mdc":\{"dbz.taskId":"0","dbz.databaseName":"main_prod","dbz.connectorName":"0feec078-acf3-47bb-a5a3-b999b8842032.b769ab5a-e00a-4e34-916e-b4f35d19ebb0","dbz.connectorType":"Postgres","dbz.connectorContext":"streaming"}

,"ndc":"","hostName":"dbz-b769ab5a-e00a-4e34-916e-b4f35d19ebb0-6db98bdb6b-bjpwr","processName":"io.debezium.server.Main","processId":1,"exception":{"refId":1,"exceptionType":"org.apache.kafka.connect.errors.DataException","message":"Invalid Java object for schema with type ARRAY: class java.lang.String for field: \"activity_items\"","frames":[

{"class":"org.apache.kafka.connect.data.ConnectSchema","method":"validateValue","line":242}

,{"class":"org.apache.kafka.connect.data.Struct","method":"put","line":216},{"class":"io.debezium.relational.TableSchemaBuilder","method":"lambda$createValueGenerator$5","line":281},{"class":"io.debezium.relational.TableSchema","method":"valueFromColumnData","line":141},{"class":"io.debezium.relational.RelationalChangeRecordEmitter","method":"emitUpdateRecord","line":102},{"class":"io.debezium.relational.RelationalChangeRecordEmitter","method":"emitChangeRecords","line":53},{"class":"io.debezium.connector.postgresql.PostgresChangeRecordEmitter","method":"emitChangeRecords","line":94},{"class":"io.debezium.pipeline.EventDispatcher","method":"dispatchDataChangeEvent","line":225},{"class":"io.debezium.connector.postgresql.PostgresStreamingChangeEventSource","method":"lambda$processMessages$0","line":279},{"class":"io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder","method":"decodeUpdate","line":479},{"class":"io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder","method":"processNotEmptyMessage","line":211},{"class":"io.debezium.connector.postgresql.connection.AbstractMessageDecoder","method":"processMessage","line":41},{"class":"io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1","method":"deserializeMessages","line":556},{"class":"io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1","method":"readPending","line":548},{"class":"io.debezium.connector.postgresql.PostgresStreamingChangeEventSource","method":"processMessages","line":216},{"class":"io.debezium.connector.postgresql.PostgresStreamingChangeEventSource","method":"execute","line":178},{"class":"io.debezium.connector.postgresql.PostgresStreamingChangeEventSource","method":"execute","line":41},{"class":"io.debezium.pipeline.ChangeEventSourceCoordinator","method":"streamEvents","line":174},{"class":"io.debezium.pipeline.ChangeEventSourceCoordinator","method":"executeChangeEventSources","line":141},{"class":"io.debezium.pipeline.ChangeEventSourceCoordinator","method":"lambda$start$0","line":109},{"class":"java.util.concurrent.Executors$RunnableAdapter","method":"call","line":515},{"class":"java.util.concurrent.FutureTask","method":"run","line":264},{"class":"java.util.concurrent.ThreadPoolExecutor","method":"runWorker","line":1128},{"class":"java.util.concurrent.ThreadPoolExecutor$Worker","method":"run","line":628},{"class":"java.lang.Thread","method":"run","line":829}]}}

 

How to reproduce the issue using our tutorial deployment?

To repro the crash:

• Make sure table replica identity is default
• Add a column type activity_items type []uuid and required
• Then insert a bunch of values to guarantee the column will be TOASTed (example below).
• Then update the row and don't update `activity_items`, see crash

 

 

DO $$

DECLARE

  uuid_array uuid[];

BEGIN

  FOR i IN 1..10000 LOOP

    uuid_array[i] = gen_random_uuid();

  END LOOP;

  UPDATE community set activity_items = uuid_array where id = '';

END $$;

 

 

Feature request or enhancement

For feature requests or enhancements, provide this information, please:

Which use case/requirement will be addressed by the proposed feature?

<Your answer>

Implementation ideas (optional)

<Your answer>