Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3989

Error processing binlog event

    XMLWordPrintable

Details

    • Task
    • Resolution: Done
    • Major
    • 1.8.0.Alpha1
    • 1.6.1.Final
    • mysql-connector
    • None

    Description

      I am running a debezium connector against an amazon MYSQL RDS with MYSQL version 5.7.26. 

      When I run the connector, I get the below error:

       

      [2021-09-09 07:32:22,048] ERROR Error during binlog processing. Last offset stored = null, binlog reader near position = mysql-bin-changelog.534024/21641 (io.debezium.conne
      ctor.mysql.MySqlStreamingChangeEventSource:1054)
      [2021-09-09 07:32:22,048] ERROR Producer failure (io.debezium.pipeline.ErrorHandler:31)
      io.debezium.DebeziumException: Error processing binlog event
              at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:366)
              at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$25(MySqlStreamingChangeEventSource.java:842)
              at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1118)
              at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:966)
              at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:606)
              at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:850)
              at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.NumberFormatException: For input string: "0 "
              at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
              at java.lang.Integer.parseInt(Integer.java:580)
              at java.lang.Integer.valueOf(Integer.java:766)
              at io.debezium.jdbc.JdbcValueConverters.lambda$convertInteger$41(JdbcValueConverters.java:892)
              at io.debezium.jdbc.JdbcValueConverters.convertValue(JdbcValueConverters.java:1288)
              at io.debezium.jdbc.JdbcValueConverters.convertInteger(JdbcValueConverters.java:880)
              at io.debezium.jdbc.JdbcValueConverters.lambda$converter$5(JdbcValueConverters.java:295)
              at io.debezium.connector.mysql.MySqlDefaultValueConverter.setColumnDefaultValue(MySqlDefaultValueConverter.java:414)
              at io.debezium.connector.mysql.antlr.listener.CreateTableParserListener.convertDefaultValueToSchemaType(CreateTableParserListener.java:147)
              at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
              at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
              at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
              at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
              at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
              at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
              at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
              at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
              at io.debezium.connector.mysql.antlr.listener.CreateTableParserListener.lambda$exitColumnCreateTable$1(CreateTableParserListener.java:75)
              at io.debezium.connector.mysql.antlr.MySqlAntlrDdlParser.runIfNotNull(MySqlAntlrDdlParser.java:357)
              at io.debezium.connector.mysql.antlr.listener.CreateTableParserListener.exitColumnCreateTable(CreateTableParserListener.java:55)
              at io.debezium.ddl.parser.mysql.generated.MySqlParser$ColumnCreateTableContext.exitRule(MySqlParser.java:4865)
              at io.debezium.antlr.ProxyParseTreeListenerUtil.delegateExitRule(ProxyParseTreeListenerUtil.java:64)
              at io.debezium.connector.mysql.antlr.listener.MySqlAntlrDdlParserListener.exitEveryRule(MySqlAntlrDdlParserListener.java:106)
              at org.antlr.v4.runtime.tree.ParseTreeWalker.exitRule(ParseTreeWalker.java:48)
              at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:30)
              at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
              at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
              at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
              at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
              at io.debezium.antlr.AntlrDdlParser.parse(AntlrDdlParser.java:87)
              at io.debezium.connector.mysql.MySqlDatabaseSchema.parseDdl(MySqlDatabaseSchema.java:213)
              at io.debezium.connector.mysql.MySqlDatabaseSchema.parseStreamingDdl(MySqlDatabaseSchema.java:200)
              at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleQueryEvent(MySqlStreamingChangeEventSource.java:571)
              at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$14(MySqlStreamingChangeEventSource.java:814)
              at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:349)
              ... 6 more
      [2021-09-09 07:32:22,049] INFO Error processing binlog event, and propagating to Kafka Connect so it stops this connector. Future binlog events read before connector is shutdown will be ignored. (io.debezium.connector.mysql.MySqlStreamingChangeEventSource:371)
      [2021-09-09 07:32:22,387] INFO 2 records sent during previous 01:15:48.0, last recorded offset: {transaction_id=null, ts_sec=1631172742, file=mysql-bin-changelog.534024, pos=21576, server_id=1771420631} (io.debezium.connector.common.BaseSourceTask:182)
      [2021-09-09 07:32:22,388] INFO WorkerSourceTask{id=orderCDC-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask:398)
      [2021-09-09 07:32:22,388] INFO WorkerSourceTask{id=orderCDC-0} flushing 2 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:415)
      [2021-09-09 07:32:22,407] INFO WorkerSourceTask{id=orderCDC-0} Finished commitOffsets successfully in 19 ms (org.apache.kafka.connect.runtime.WorkerSourceTask:497)
      [2021-09-09 07:32:22,407] ERROR WorkerSourceTask{id=orderCDC-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:179)
      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
              at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)
              at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:366)
              at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$25(MySqlStreamingChangeEventSource.java:842)
              at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1118)
              at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:966)
              at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:606)
              at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:850)
              at java.lang.Thread.run(Thread.java:748)
      

      binlog mysql-bin-changelog.534024/21641 content:

       

       

         Log_name: mysql-bin-changelog.534024
              Pos: 21641
       Event_type: Query
        Server_id: 1771420631
      End_log_pos: 22602
        Info: use `bbt`; create table wms_abcd(
        `id` int(10) NOT NULL DEFAULT 0,
        `foo` varchar(32) NOT NULL DEFAULT '',
        `bar` varchar(25) NOT NULL DEFAULT '',
        `foobar` int(11)   NOT NULL default '0 ',
        PRIMARY KEY (`id`),
        KEY `idx_foo_bar` (`foo`,`bar`)
      )ENGINE=InnoDB  DEFAULT CHARSET=utf8
      
      

      I find the field `foobar` default value contains spaces, did this problem cause the task to fail? how should i solve this.

       

      Attachments

        Issue Links

          Activity

            People

              blcksrx Hossein Torabi
              xxm404 404 xxm (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: