Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2450

Postgres connector does not handle -infinity timestamp values

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • None
    • None
    • postgresql-connector
    • None
    • Hide

      1) Install postgres (any version) using either wal2json or pgoutput (untested with decoderbufs)

      2) Create a table with a column of type "timestamp without time zone"

      3) Turn on debezium and listen to changes to that table

      4) Insert a record with a timestamp value of "-infinity"

      5) Debezium crashes as its unable to parse "-infinity"

       

      Show
      1) Install postgres (any version) using either wal2json or pgoutput (untested with decoderbufs) 2) Create a table with a column of type "timestamp without time zone" 3) Turn on debezium and listen to changes to that table 4) Insert a record with a timestamp value of "-infinity" 5) Debezium crashes as its unable to parse "-infinity"  

    Description

      Postgres connector fails when parsing "-infinity" timestamps.

       

      2020-08-25 18:53:51,818 ERROR  Postgres|cdc.perl|postgres-connector-task  Cannot parse time/date value '-infinity', expected format 'y..y-MM-dd HH:mm:ss[.S]'   [io.debezium.connector.postgresql.connection.wal2json.DateTimeFormat$ISODateTimeFormat]2020-08-25 18:53:51,818 ERROR  Postgres|cdc.perl|postgres-connector-task  Cannot parse time/date value '-infinity', expected format 'y..y-MM-dd HH:mm:ss[.S]'   [io.debezium.connector.postgresql.connection.wal2json.DateTimeFormat$ISODateTimeFormat]2020-08-25 18:53:51,819 ERROR  Postgres|cdc.perl|postgres-connector-task  Producer failure   [io.debezium.pipeline.ErrorHandler]org.apache.kafka.connect.errors.ConnectException: Error while processing event at offset {transaction_id=null, lsn_proc=7501158936, lsn_commit=7501158936, lsn=7501158936, txId=10850, ts_usec=1598384664160332} at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:220) at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.lambda$processMessages$0(PostgresStreamingChangeEventSource.java:214) at io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder.decodeInsert(PgOutputMessageDecoder.java:380) at io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder.processNotEmptyMessage(PgOutputMessageDecoder.java:191) at io.debezium.connector.postgresql.connection.AbstractMessageDecoder.processMessage(AbstractMessageDecoder.java:42) at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.deserializeMessages(PostgresReplicationConnection.java:480) at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.readPending(PostgresReplicationConnection.java:472) at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.processMessages(PostgresStreamingChangeEventSource.java:172) at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:139) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:108) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832)Caused by: org.apache.kafka.connect.errors.ConnectException: java.time.format.DateTimeParseException: Text '-infinity' could not be parsed at index 0 at io.debezium.connector.postgresql.connection.wal2json.DateTimeFormat$ISODateTimeFormat.format(DateTimeFormat.java:143) at io.debezium.connector.postgresql.connection.wal2json.DateTimeFormat$ISODateTimeFormat.timestampToInstant(DateTimeFormat.java:149) at io.debezium.connector.postgresql.connection.AbstractColumnValue.asInstant(AbstractColumnValue.java:68) at io.debezium.connector.postgresql.connection.ReplicationMessageColumnValueResolver.resolveValue(ReplicationMessageColumnValueResolver.java:111) at io.debezium.connector.postgresql.connection.pgoutput.PgOutputReplicationMessage.getValue(PgOutputReplicationMessage.java:96) at io.debezium.connector.postgresql.connection.pgoutput.PgOutputMessageDecoder$1.getValue(PgOutputMessageDecoder.java:579) at io.debezium.connector.postgresql.PostgresChangeRecordEmitter.columnValues(PostgresChangeRecordEmitter.java:170) at io.debezium.connector.postgresql.PostgresChangeRecordEmitter.getNewColumnValues(PostgresChangeRecordEmitter.java:113) at io.debezium.relational.RelationalChangeRecordEmitter.emitCreateRecord(RelationalChangeRecordEmitter.java:64) at io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:45) at io.debezium.connector.postgresql.PostgresChangeRecordEmitter.emitChangeRecords(PostgresChangeRecordEmitter.java:88) at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:193) ... 14 moreCaused by: java.time.format.DateTimeParseException: Text '-infinity' could not be parsed at index 0 at java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2050) at java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1878) at io.debezium.connector.postgresql.connection.wal2json.DateTimeFormat$ISODateTimeFormat.lambda$timestampToInstant$3(DateTimeFormat.java:149) at io.debezium.connector.postgresql.connection.wal2json.DateTimeFormat$ISODateTimeFormat.format(DateTimeFormat.java:139) ... 25 more2020-08-25 18:53:51,823 INFO   Postgres|cdc.perl|postgres-connector-task  Finished streaming   [io.debezium.pipeline.ChangeEventSourceCoordinator]

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem that impairs or prevents the functions of the product. DBZ-2565 decoderbufs Segmentation fault on timestamp with infinity

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is related to

          Feature Request - Feature requests from customers and/or users DBZ-2614 Process special values in temporal datatypes

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              Unassigned Unassigned
              parbmatthews Brady Matthews (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: