Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3469

"Found null value for non-optional schema" error when issuing TRUNCATE from Postgres on a table with a PK

XMLWordPrintable

    • False
    • False
    • Undefined
    • Hide

      0. Start debezium container:

           KEY_CONVERTER: io.confluent.connect.avro.AvroConverter
           VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter 

      1 Configure Debezium:

      { "name": "psql-connector",
       "config":
      { "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "database.hostname": "postgres", "database.port": "5432", "database.user": "debezium", "database.password": "debezium", "database.dbname" : "postgres", "database.server.name": "postgres", "plugin.name": "pgoutput", "slot.name" : "tester", "database.history.kafka.bootstrap.servers": "kafka:9092", "database.history.kafka.topic": "schema-changes.history", "truncate.handling.mode": "include", "provide.transaction.metadata": "true" }
      }

      2. Run in Postgres:

      CREATE TABLE truncate_table (f1 INTEGER PRIMARY KEY);
      INSERT INTO truncate_table VALUES (1),(2),(3);
      TRUNCATE TABLE truncate_table;\{code}
      
      Show
      0. Start debezium container: KEY_CONVERTER: io.confluent.connect.avro.AvroConverter VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter 1 Configure Debezium: { "name" : "psql-connector" , "config" : { "connector.class" : "io.debezium.connector.postgresql.PostgresConnector" , "database.hostname" : "postgres" , "database.port" : "5432" , "database.user" : "debezium" , "database.password" : "debezium" , "database.dbname" : "postgres" , "database.server.name" : "postgres" , "plugin.name" : "pgoutput" , "slot.name" : "tester" , "database.history.kafka.bootstrap.servers" : "kafka:9092" , "database.history.kafka.topic" : "schema-changes.history" , "truncate.handling.mode" : "include" , "provide.transaction.metadata" : " true " } } 2. Run in Postgres: CREATE TABLE truncate_table (f1 INTEGER PRIMARY KEY); INSERT INTO truncate_table VALUES (1),(2),(3); TRUNCATE TABLE truncate_table;\{code}

      Postgres version: 11.4
      Debezium version: debezium/connect:1.5, from container

      Attempting to run TRUNCATE in Postgres results in the following error in Debezium:

      2021-04-26 11:28:56,958 ERROR  ||  WorkerSourceTask{id=psql-connector-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted   [org.apache.kafka.connect.runtime.WorkerTask]
      org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
      	at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:206)
      	at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execute(RetryWithToleranceOperator.java:132)
      	at org.apache.kafka.connect.runtime.WorkerSourceTask.convertTransformedRecord(WorkerSourceTask.java:311)
      	at org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:340)
      	at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:264)
      	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:185)
      	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:234)
      	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      	at java.base/java.lang.Thread.run(Thread.java:834)
      Caused by: org.apache.kafka.connect.errors.DataException: Found null value for non-optional schema
      	at io.confluent.connect.avro.AvroData.validateSchemaValue(AvroData.java:1177)
      	at io.confluent.connect.avro.AvroData.fromConnectData(AvroData.java:381)
      	at io.confluent.connect.avro.AvroData.fromConnectData(AvroData.java:344)
      	at io.confluent.connect.avro.AvroConverter.fromConnectData(AvroConverter.java:89)
      	at org.apache.kafka.connect.storage.Converter.fromConnectData(Converter.java:63)
      	at org.apache.kafka.connect.runtime.WorkerSourceTask.lambda$convertTransformedRecord$1(WorkerSourceTask.java:311)
      	at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:156)
      	at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:190)
      	... 11 more
      2021-04-26 11:28:56,959 INFO   ||  Stopping down connector   [io.debezium.connector.common.BaseSourceTask]
      

              jpechane Jiri Pechanec
              philip_stoev Philip Stoev (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: