Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-9558

Timestamp values cause NPE in Postgres on JDK23+

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 3.3.2.Final, 3.4.0.Alpha1
    • 3.2.0.Final, 3.3.0.Final
    • None
    • None

      Bug report

      When running on JDK23 and newer, sometimes timestamp values from Postgres would cause NPE:

      05:06:00.609 [debezium-postgresconnector-pubs-change-event-source-coordinator] ERROR io.debezium.relational.TableSchemaBuilder - Failed to properly convert data value for 'public.titles.pubdate' of type timestamp
          java.lang.NullPointerException: Cannot invoke "java.time.chrono.ChronoLocalDate.toEpochDay()" because the return value of "java.time.chrono.ChronoLocalDateTime.toLocalDate()" is null
              at java.base/java.time.chrono.ChronoLocalDateTime.toEpochSecond(ChronoLocalDateTime.java:472)
              at java.base/java.time.chrono.ChronoLocalDateTime.toInstant(ChronoLocalDateTime.java:453)
              at io.debezium.time.Timestamp.toEpochMillis(Timestamp.java:77)
              at io.debezium.jdbc.JdbcValueConverters.lambda$convertTimestampToEpochMillisAsDate$26(JdbcValueConverters.java:545)
              at io.debezium.jdbc.JdbcValueConverters.convertValue(JdbcValueConverters.java:1415)
              at io.debezium.jdbc.JdbcValueConverters.convertTimestampToEpochMillisAsDate(JdbcValueConverters.java:543)
              at io.debezium.jdbc.JdbcValueConverters.convertTimestamp(JdbcValueConverters.java:448)
              at io.debezium.jdbc.JdbcValueConverters.lambda$converter$15(JdbcValueConverters.java:310)
              at io.debezium.relational.ValueConverter.lambda$and$1(ValueConverter.java:50)
              at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$5(TableSchemaBuilder.java:327)
              at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:142)
              at io.debezium.relational.RelationalChangeRecordEmitter.emitCreateRecord(RelationalChangeRecordEmitter.java:71)
              at io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:47)
              at io.debezium.connector.postgresql.PostgresChangeRecordEmitter.emitChangeRecords(PostgresChangeRecordEmitter.java:93)
              at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:290)
              at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.processReplicationMessages(PostgresStreamingChangeEventSource.java:344)
              at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.lambda$processMessages$0(PostgresStreamingChangeEventSource.java:243)
              at io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder.decodeInsert(PgOutputMessageDecoder.java:432)
              at io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder.processNotEmptyMessage(PgOutputMessageDecoder.java:208)
              at io.debezium.connector.postgresql.connection.AbstractMessageDecoder.processMessage(AbstractMessageDecoder.java:41)
              at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.deserializeMessages(PostgresReplicationConnection.java:799)
              at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.readPending(PostgresReplicationConnection.java:791)
              at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.processMessages(PostgresStreamingChangeEventSource.java:243)
              at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:194)
              at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:49)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:326)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:207)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:147)
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:545)
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:328)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090)
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
              at java.base/java.lang.Thread.run(Thread.java:1474)
      

      What Debezium connector do you use and what version?

      postgres:3.2.0.Final

      also reproducible on postgres:3.3.0.Final

      What is the connector configuration?

      connector.class: "io.debezium.connector.postgresql.PostgresConnector"
      plugin.name: "pgoutput"
      publication.autocreate.mode: "filtered"
      publication.name: "relational_migrator_publication"
      slot.name: "${jobId}"
      transforms: "${timescaledbTransformsName}"
      transforms.timescaledb.type: "io.debezium.connector.postgresql.transforms.timescaledb.TimescaleDb"
      transforms.timescaledb.database.hostname: "${databaseHostname}"
      transforms.timescaledb.database.port: "${databasePort}"
      transforms.timescaledb.database.user: "${databaseUser}"
      transforms.timescaledb.database.password: "${databasePassword}"
      transforms.timescaledb.database.dbname: "${databaseName}"
      

      What is the captured database version and mode of deployment?

      (E.g. on-premises, with a specific cloud provider, etc.)

      postgres:15 in Docker (via testcontainers)

      What behavior do you expect?

      Given a table definition:

      create table pubs.public.employee
      (
          emp_id    char(9)                      not null
              constraint PK_emp_id
                  primary key
              constraint CK_emp_id
                  check (emp_id ~ '[A-Z][A-Z][A-Z][1-9][0-9][0-9][0-9][0-9][FM]' OR
              emp_id ~ '[A-Z]-[A-Z][1-9][0-9][0-9][0-9][0-9][FM]'),
          fname     varchar(20)                not null,
          minit     char,
          lname     varchar(30)                not null,
          job_id    smallint default 1         not null
              references jobs ON DELETE CASCADE,
          job_lvl   smallint  default 10,
        -- foreign keys do not seem to matter in this scenario
          pub_id    char(4)  default '9952'    not null
              references publishers ON DELETE CASCADE,
          hire_date timestamp default now() not null
      );
      

      and data

      INSERT INTO pubs.public.employee (emp_id,fname,minit,lname,job_id,job_lvl,pub_id,hire_date) VALUES ('CFH28514M','Carlos','F','Hernadez',5,211,'9999','1989-04-21 00:00:00.0');
      

      i expect the hire_date to contain a valid date, 1989-04-21 and time 00:00:00 components.

      What behavior do you see?

      this works just fine on Java 17, 21 and 22.

      once the same setup switched to JDK 23, a NullPointerException is thrown.

      i was only able to reproduce it on CI (Github Actions), but not locally.

      enforcing timezone on a Postgres container has little to no effect.

      Do you see the same behaviour using the latest released Debezium version?

      yes

      Do you have the connector logs, ideally from start till finish?

      see above plus this zullip thread

      How to reproduce the issue using our tutorial deployment?

      n/a

              Unassigned Unassigned
              artem.shubovych Artem Shubovych
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: