Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1325

Debezium PSQL connector disconnecting at regular intervals and causing loss in data.

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • None
    • None
    • postgresql-connector
    • None

    Description

      Hi,
      I am using Debezium PSQL connector for capturing CDC. In one of the connectors i am using, the following error is frequently occurring causing connector disconnection . I have also observed loss of change events in the cdc. Can any one please help me out in understanding why this issue is occurring and how i can resolve it.

      org.apache.kafka.connect.errors.ConnectException: An exception ocurred in the change event producer. This connector will be stopped.
      	at io.debezium.connector.base.ChangeEventQueue.throwProducerFailureIfPresent(ChangeEventQueue.java:170)
      	at io.debezium.connector.base.ChangeEventQueue.poll(ChangeEventQueue.java:151)
      	at io.debezium.connector.postgresql.PostgresConnectorTask.poll(PostgresConnectorTask.java:161)
      	at org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:244)
      	at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:220)
      	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)
      	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: org.postgresql.util.PSQLException: Database connection failed when writing to copy
      	at org.postgresql.core.v3.QueryExecutorImpl.flushCopy(QueryExecutorImpl.java:1015)
      	at org.postgresql.core.v3.CopyDualImpl.flushCopy(CopyDualImpl.java:23)
      	at org.postgresql.core.v3.replication.V3PGReplicationStream.updateStatusInternal(V3PGReplicationStream.java:190)
      	at org.postgresql.core.v3.replication.V3PGReplicationStream.timeUpdateStatus(V3PGReplicationStream.java:181)
      	at org.postgresql.core.v3.replication.V3PGReplicationStream.readInternal(V3PGReplicationStream.java:121)
      	at org.postgresql.core.v3.replication.V3PGReplicationStream.read(V3PGReplicationStream.java:70)
      	at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.read(PostgresReplicationConnection.java:264)
      	at io.debezium.connector.postgresql.RecordsStreamProducer.streamChanges(RecordsStreamProducer.java:140)
      	at io.debezium.connector.postgresql.RecordsStreamProducer.lambda$start$0(RecordsStreamProducer.java:123)
      	... 5 more
      Caused by: java.net.SocketException: Connection reset
      	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115)
      	at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
      	at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
      	at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
      	at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:879)
      	at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:850)
      	at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
      	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
      	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
      	at org.postgresql.core.PGStream.flush(PGStream.java:527)
      	at org.postgresql.core.v3.QueryExecutorImpl.flushCopy(QueryExecutorImpl.java:1012)
      	... 13 more
      
      

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              yanamadala.prakash@oyorooms.com Yanamadala JaiPrakash (Inactive)
              Votes:
              2 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: