-
Bug
-
Resolution: Done
-
Major
-
3.2.0.Final, 3.3.0.Final
-
None
-
None
-
False
-
-
False
Bug report
When running on JDK23 and newer, sometimes timestamp values from Postgres would cause NPE:
05:06:00.609 [debezium-postgresconnector-pubs-change-event-source-coordinator] ERROR io.debezium.relational.TableSchemaBuilder - Failed to properly convert data value for 'public.titles.pubdate' of type timestamp java.lang.NullPointerException: Cannot invoke "java.time.chrono.ChronoLocalDate.toEpochDay()" because the return value of "java.time.chrono.ChronoLocalDateTime.toLocalDate()" is null at java.base/java.time.chrono.ChronoLocalDateTime.toEpochSecond(ChronoLocalDateTime.java:472) at java.base/java.time.chrono.ChronoLocalDateTime.toInstant(ChronoLocalDateTime.java:453) at io.debezium.time.Timestamp.toEpochMillis(Timestamp.java:77) at io.debezium.jdbc.JdbcValueConverters.lambda$convertTimestampToEpochMillisAsDate$26(JdbcValueConverters.java:545) at io.debezium.jdbc.JdbcValueConverters.convertValue(JdbcValueConverters.java:1415) at io.debezium.jdbc.JdbcValueConverters.convertTimestampToEpochMillisAsDate(JdbcValueConverters.java:543) at io.debezium.jdbc.JdbcValueConverters.convertTimestamp(JdbcValueConverters.java:448) at io.debezium.jdbc.JdbcValueConverters.lambda$converter$15(JdbcValueConverters.java:310) at io.debezium.relational.ValueConverter.lambda$and$1(ValueConverter.java:50) at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$5(TableSchemaBuilder.java:327) at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:142) at io.debezium.relational.RelationalChangeRecordEmitter.emitCreateRecord(RelationalChangeRecordEmitter.java:71) at io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:47) at io.debezium.connector.postgresql.PostgresChangeRecordEmitter.emitChangeRecords(PostgresChangeRecordEmitter.java:93) at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:290) at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.processReplicationMessages(PostgresStreamingChangeEventSource.java:344) at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.lambda$processMessages$0(PostgresStreamingChangeEventSource.java:243) at io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder.decodeInsert(PgOutputMessageDecoder.java:432) at io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder.processNotEmptyMessage(PgOutputMessageDecoder.java:208) at io.debezium.connector.postgresql.connection.AbstractMessageDecoder.processMessage(AbstractMessageDecoder.java:41) at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.deserializeMessages(PostgresReplicationConnection.java:799) at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.readPending(PostgresReplicationConnection.java:791) at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.processMessages(PostgresStreamingChangeEventSource.java:243) at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:194) at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:49) at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:326) at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:207) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:147) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:545) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:328) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614) at java.base/java.lang.Thread.run(Thread.java:1474)
What Debezium connector do you use and what version?
postgres:3.2.0.Final
also reproducible on postgres:3.3.0.Final
What is the connector configuration?
connector.class: "io.debezium.connector.postgresql.PostgresConnector" plugin.name: "pgoutput" publication.autocreate.mode: "filtered" publication.name: "relational_migrator_publication" slot.name: "${jobId}" transforms: "${timescaledbTransformsName}" transforms.timescaledb.type: "io.debezium.connector.postgresql.transforms.timescaledb.TimescaleDb" transforms.timescaledb.database.hostname: "${databaseHostname}" transforms.timescaledb.database.port: "${databasePort}" transforms.timescaledb.database.user: "${databaseUser}" transforms.timescaledb.database.password: "${databasePassword}" transforms.timescaledb.database.dbname: "${databaseName}"
What is the captured database version and mode of deployment?
(E.g. on-premises, with a specific cloud provider, etc.)
postgres:15 in Docker (via testcontainers)
What behavior do you expect?
Given a table definition:
create table pubs.public.employee ( emp_id char(9) not null constraint PK_emp_id primary key constraint CK_emp_id check (emp_id ~ '[A-Z][A-Z][A-Z][1-9][0-9][0-9][0-9][0-9][FM]' OR emp_id ~ '[A-Z]-[A-Z][1-9][0-9][0-9][0-9][0-9][FM]'), fname varchar(20) not null, minit char, lname varchar(30) not null, job_id smallint default 1 not null references jobs ON DELETE CASCADE, job_lvl smallint default 10, -- foreign keys do not seem to matter in this scenario pub_id char(4) default '9952' not null references publishers ON DELETE CASCADE, hire_date timestamp default now() not null );
and data
INSERT INTO pubs.public.employee (emp_id,fname,minit,lname,job_id,job_lvl,pub_id,hire_date) VALUES ('CFH28514M','Carlos','F','Hernadez',5,211,'9999','1989-04-21 00:00:00.0');
i expect the hire_date to contain a valid date, 1989-04-21 and time 00:00:00 components.
What behavior do you see?
this works just fine on Java 17, 21 and 22.
once the same setup switched to JDK 23, a NullPointerException is thrown.
i was only able to reproduce it on CI (Github Actions), but not locally.
enforcing timezone on a Postgres container has little to no effect.
Do you see the same behaviour using the latest released Debezium version?
yes
Do you have the connector logs, ideally from start till finish?
see above plus this zullip thread
How to reproduce the issue using our tutorial deployment?
n/a