Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-477

Postgres connector can't get change stream

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Critical Critical
    • None
    • 0.6.2
    • postgresql-connector
    • None

      Dear,

      When I run debezium-embedded demo on my machine, strange that every time I first update my table I got postgresql update successfully but with a warning message, also with an exception throwed by debezium. But then I update there is none postgres warning and no reaction by connector. Here is my configuration and error message from postgresql and demo program.

      Configuration:

      Configuration config = Configuration.create()
                      /* begin engine properties */
                      .with("connector.class", "io.debezium.connector.postgresql.PostgresConnector")//
                      .with("offset.storage", "org.apache.kafka.connect.storage.FileOffsetBackingStore")//
                      .with("offset.storage.file.filename", path + "/debezium/offset.dat")//
                      .with("database.history", "io.debezium.relational.history.FileDatabaseHistory")//
                      .with("database.history.file.filename", path + "/debezium/dbhistory.dat")//
                      .with("offset.flush.interval.ms", 60000)//
                      .with("name", "inventory-connector1")//
                      .with("database.hostname", "192.168.0.38")//
                      .with("database.port", 5432)//
                      .with("database.user", "postgres")//
                      .with("database.password", "postgres")//
                      .with("database.dbname", "postgres")//
                      .with("database.server.name", "postgres")//
                      .with("table.blacklist", "public.redis_db0")//
                      .with("bootstrap.servers", "192.168.0.38:9092")//
                      .with("poll.interval.ms", "5000")//
                      .build();
      

      postgresql warning message:

      [SQL]
      update customers set last_name='hello' where id=1;
      
      WARNING:  terminating connection because of crash of another server process
      DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
      HINT:  In a moment you should be able to reconnect to the database and repeat your command.
      
      Affect rows: 1
      

      postgres connector error message:

      2017-11-21 18:23:23.437 [pool-2-thread-1] ERROR io.debezium.embedded.EmbeddedEngine - Error while trying to run connector class 'io.debezium.connector.postgresql.PostgresConnector'
      org.apache.kafka.connect.errors.ConnectException: org.postgresql.util.PSQLException: Database connection failed when reading from copy
      	at io.debezium.connector.postgresql.PostgresConnectorTask.poll(PostgresConnectorTask.java:165) ~[debezium-connector-postgres-0.6.2.jar:0.6.2]
      	at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:647) ~[debezium-embedded-0.6.2.jar:0.6.2]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:1.8.0_131]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.8.0_131]
      	at java.lang.Thread.run(Unknown Source) ~[na:1.8.0_131]
      Caused by: org.postgresql.util.PSQLException: Database connection failed when reading from copy
      	at org.postgresql.core.v3.QueryExecutorImpl.readFromCopy(QueryExecutorImpl.java:1028) ~[postgresql-42.1.4.jar:42.1.4]
      	at org.postgresql.core.v3.CopyDualImpl.readFromCopy(CopyDualImpl.java:41) ~[postgresql-42.1.4.jar:42.1.4]
      	at org.postgresql.core.v3.replication.V3PGReplicationStream.receiveNextData(V3PGReplicationStream.java:155) ~[postgresql-42.1.4.jar:42.1.4]
      	at org.postgresql.core.v3.replication.V3PGReplicationStream.readInternal(V3PGReplicationStream.java:124) ~[postgresql-42.1.4.jar:42.1.4]
      	at org.postgresql.core.v3.replication.V3PGReplicationStream.read(V3PGReplicationStream.java:70) ~[postgresql-42.1.4.jar:42.1.4]
      	at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.read(PostgresReplicationConnection.java:176) ~[debezium-connector-postgres-0.6.2.jar:0.6.2]
      	at io.debezium.connector.postgresql.RecordsStreamProducer.streamChanges(RecordsStreamProducer.java:112) ~[debezium-connector-postgres-0.6.2.jar:0.6.2]
      	at io.debezium.connector.postgresql.RecordsStreamProducer.lambda$start$1(RecordsStreamProducer.java:98) ~[debezium-connector-postgres-0.6.2.jar:0.6.2]
      	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.8.0_131]
      	at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.8.0_131]
      	... 3 common frames omitted
      Caused by: java.io.EOFException: null
      	at org.postgresql.core.PGStream.receiveChar(PGStream.java:290) ~[postgresql-42.1.4.jar:42.1.4]
      	at org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1070) ~[postgresql-42.1.4.jar:42.1.4]
      	at org.postgresql.core.v3.QueryExecutorImpl.readFromCopy(QueryExecutorImpl.java:1026) ~[postgresql-42.1.4.jar:42.1.4]
      	... 12 common frames omitted
      

      Can anybody resolve this? Thanks.

            jpechane Jiri Pechanec
            wendaoshuangyue Sun Victor (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: