-
Bug
-
Resolution: Done
-
Major
-
1.9.0.Final
-
None
-
False
-
-
False
What Debezium connector do you use and what version?
https://hub.docker.com/r/debezium/connect/tags
Image: debezium/connect:1.9
What is the connector configuration?
{
"name": "inventory-connector-dev",
"config": {
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"tasks.max": "1",
"database.hostname": "hostname",
"database.port": "5432",
"database.user": "user",
"database.password": "password",
"database.dbname" : "dbName",
"database.server.name": "postgre-service",
"table.include.list": "ext.(.*)",
"plugin.name": "pgoutput",
"snapshot.mode": "initial",
"transforms": "AddPrefix",
"transforms.AddPrefix.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.AddPrefix.regex": "dmp.ext.(.*)",
"transforms.AddPrefix.replacement": "data.cdc.dbname"
}
}
What is the captured database version and mode of depoyment?
(E.g. on-premises, with a specific cloud provider, etc.)
On-premises, DBMS: PostgreSQL (ver. 10.21 (Debian 10.21-1.pgdg90+1))
What behaviour do you see?
Getting the following error:
2022-07-12 10:25:05,289 ERROR Postgres|postgre-service|snapshot Producer failure [io.debezium.pipeline.ErrorHandler] io.debezium.DebeziumException: java.lang.IllegalArgumentException: The column "COALESCE(group_id, ''::text)" is referenced as PRIMARY KEY, but a matching column is not defined in table "ext.counter"! at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85) at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155) at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.IllegalArgumentException: The column "COALESCE(group_id, ''::text)" is referenced as PRIMARY KEY, but a matching column is not defined in table "ext.counter"! at io.debezium.relational.TableEditorImpl.lambda$updatePrimaryKeys$0(TableEditorImpl.java:106) at java.base/java.util.ArrayList.removeIf(ArrayList.java:1702) at java.base/java.util.ArrayList.removeIf(ArrayList.java:1690) at io.debezium.relational.TableEditorImpl.updatePrimaryKeys(TableEditorImpl.java:102) at io.debezium.relational.TableEditorImpl.create(TableEditorImpl.java:267) at io.debezium.relational.Tables.lambda$overwriteTable$2(Tables.java:192) at io.debezium.util.FunctionalReadWriteLock.write(FunctionalReadWriteLock.java:84) at io.debezium.relational.Tables.overwriteTable(Tables.java:186) at io.debezium.jdbc.JdbcConnection.readSchema(JdbcConnection.java:1209) at io.debezium.connector.postgresql.PostgresSchema.refresh(PostgresSchema.java:87) at io.debezium.connector.postgresql.PostgresSnapshotChangeEventSource.connectionCreated(PostgresSnapshotChangeEventSource.java:92) at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:95) at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76) ... 8 more
DDL of the table:
create table ext.counter
(
campaign_id text not null
references ext.campaign
on update cascade on delete cascade,
group_id text
references ext.group
on update cascade on delete cascade,
sent_cnt integer default 0,
time_sent_cnt integer default 0,
last_sent_dt timestamp default LOCALTIMESTAMP,
emd_ins_dt timestamp default LOCALTIMESTAMP not null,
emd_upd_dt timestamp
);
create unique index counter_ukey
on ext.counter (campaign_id, COALESCE(group_id, ''::text));
