Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-273

Connect doesn't reconnect to PostgreSQL if connection breaks

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • None
    • 0.5
    • postgresql-connector
    • None
    • Hide

      Use Debezium/Connect with PostgreSQL (setup from the tutorial is OK). Start Postgres and Connect, start extracing events then restart Postgres.

      Show
      Use Debezium/Connect with PostgreSQL (setup from the tutorial is OK). Start Postgres and Connect, start extracing events then restart Postgres.

    Description

      Summary

      If Connector (and the underlying JDBC driver) loses connection to Postgres, then it won't reconnect.

      Steps to reproduce
      Use Debezium/Connect with PostgreSQL (setup from the tutorial is OK). Start Postgres and Connect, start extracing events then restart Postgres.

      Expected behavior
      CDC continues to work.

      Actual behavior
      CDC stops working with something like:

      org.apache.kafka.connect.errors.ConnectException: org.postgresql.util.PSQLException: Database connection failed when writing to copy
        at io.debezium.connector.postgresql.RecordsStreamProducer.commit(RecordsStreamProducer.java:133)
        at io.debezium.connector.postgresql.PostgresConnectorTask.commit(PostgresConnectorTask.java:154)
        at org.apache.kafka.connect.runtime.WorkerSourceTask.commitSourceTask(WorkerSourceTask.java:381)
        at org.apache.kafka.connect.runtime.WorkerSourceTask.commitOffsets(WorkerSourceTask.java:328)
        at org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter.commit(SourceTaskOffsetCommitter.java:109)
        at org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter.access$000(SourceTaskOffsetCommitter.java:46)
        at org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter$1.run(SourceTaskOffsetCommitter.java:83)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.postgresql.util.PSQLException: Database connection failed when writing to copy
        at org.postgresql.core.v3.QueryExecutorImpl.flushCopy(QueryExecutorImpl.java:942)
        at org.postgresql.core.v3.CopyDualImpl.flushCopy(CopyDualImpl.java:23)
        at org.postgresql.core.v3.replication.V3PGReplicationStream.updateStatusInternal(V3PGReplicationStream.java:176)
        at org.postgresql.core.v3.replication.V3PGReplicationStream.forceUpdateStatus(V3PGReplicationStream.java:99)
        at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.flushLSN(PostgresReplicationConnection.java:214)
        at io.debezium.connector.postgresql.RecordsStreamProducer.commit(RecordsStreamProducer.java:128)
        ... 13 more
Caused by: java.net.SocketException: Broken pipe (Write failed)
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
        at org.postgresql.core.PGStream.flush(PGStream.java:553)
        at org.postgresql.core.v3.QueryExecutorImpl.flushCopy(QueryExecutorImpl.java:939)
        ... 18 more

      Attachments

        Issue Links

          duplicates

          Feature Request - Feature requests from customers and/or users DBZ-1723 Retry polling on configured exceptions

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is related to

          Bug - A problem that impairs or prevents the functions of the product. DBZ-1846 PostgreSQL Connector java.net.SocketException: Broken pipe (Write failed)

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. DBZ-281 Postgres task should fail when connection to server is lost

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          relates to

          Feature Request - Feature requests from customers and/or users DBZ-1723 Retry polling on configured exceptions

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              jpechane Jiri Pechanec
              matevarga Mate Varga (Inactive)
              Votes:
              7 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: