Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-189

DBZ Errors while adding a json column table name.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 0.4
    • 0.3.6
    • mysql-connector
    • None
      1. Add a table with json data type.
      2. Insert record
      3. Start DBZ
      4. Update JSON in record
      5. Should see this error.

      We added a table which has json column in it.
      DBZ mysql connector fails while laoding the data.

      Here is the stack trace of the issue

      [2017-02-08 21:17:39,941] ERROR Failed due to error: Error processing binlog event (io.debezium.connector.mysql.BinlogReader:143)
      org.apache.kafka.connect.errors.ConnectException: Failed to parse and read a JSON value on settings JSON: The key offset in the JSON document is 42752 and is too big for the binary form of the document (34560)
      	at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:164)
      	at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:142)
      	at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:324)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:877)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:735)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:451)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient$5.run(BinaryLogClient.java:632)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to parse and read a JSON value on settings JSON: The key offset in the JSON document is 42752 and is too big for the binary form of the document (34560)
      	at io.debezium.connector.mysql.MySqlValueConverters.convertJson(MySqlValueConverters.java:209)
      	at io.debezium.connector.mysql.MySqlValueConverters.lambda$converter$30(MySqlValueConverters.java:117)
      	at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$170(TableSchemaBuilder.java:230)
      	at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:111)
      	at io.debezium.connector.mysql.RecordMakers$1.update(RecordMakers.java:232)
      	at io.debezium.connector.mysql.RecordMakers$RecordsForTable.update(RecordMakers.java:444)
      	at io.debezium.connector.mysql.BinlogReader.handleUpdate(BinlogReader.java:547)
      	at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:311)
      	... 5 more
      Caused by: java.io.IOException: The key offset in the JSON document is 42752 and is too big for the binary form of the document (34560)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.readUnsignedIndex(JsonBinary.java:867)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseObject(JsonBinary.java:334)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:216)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseObject(JsonBinary.java:398)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:216)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseObject(JsonBinary.java:398)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:216)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:210)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:181)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseAsString(JsonBinary.java:168)
      	at io.debezium.connector.mysql.MySqlValueConverters.convertJson(MySqlValueConverters.java:206)
      	... 12 more
      [2017-02-08 21:17:39,941] ERROR Failed due to error: Error processing binlog event (io.debezium.connector.mysql.BinlogReader)
      org.apache.kafka.connect.errors.ConnectException: Failed to parse and read a JSON value on settings JSON: The key offset in the JSON document is 42752 and is too big for the binary form of the document (34560)
      	at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:164)
      	at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:142)
      	at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:324)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:877)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:735)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:451)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient$5.run(BinaryLogClient.java:632)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to parse and read a JSON value on settings JSON: The key offset in the JSON document is 42752 and is too big for the binary form of the document (34560)
      	at io.debezium.connector.mysql.MySqlValueConverters.convertJson(MySqlValueConverters.java:209)
      	at io.debezium.connector.mysql.MySqlValueConverters.lambda$converter$30(MySqlValueConverters.java:117)
      	at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$170(TableSchemaBuilder.java:230)
      	at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:111)
      	at io.debezium.connector.mysql.RecordMakers$1.update(RecordMakers.java:232)
      	at io.debezium.connector.mysql.RecordMakers$RecordsForTable.update(RecordMakers.java:444)
      	at io.debezium.connector.mysql.BinlogReader.handleUpdate(BinlogReader.java:547)
      	at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:311)
      	... 5 more
      Caused by: java.io.IOException: The key offset in the JSON document is 42752 and is too big for the binary form of the document (34560)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.readUnsignedIndex(JsonBinary.java:867)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseObject(JsonBinary.java:334)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:216)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseObject(JsonBinary.java:398)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:216)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseObject(JsonBinary.java:398)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:216)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:210)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:181)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseAsString(JsonBinary.java:168)
      	at io.debezium.connector.mysql.MySqlValueConverters.convertJson(MySqlValueConverters.java:206)
      	... 12 more
      [2017-02-08 21:17:39,944] INFO Error processing binlog event, and propagating to Kafka Connect so it stops this connector. Future binlog events read before connector is shutdown will be ignored. (io.debezium.connector.mysql.BinlogReader:329)
      [2017-02-08 21:17:39,944] INFO Error processing binlog event, and propagating to Kafka Connect so it stops this connector. Future binlog events read before connector is shutdown will be ignored. (io.debezium.connector.mysql.BinlogReader)
      [2017-02-08 21:17:39,944] ERROR Task debizium-connector-llama-0 threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:142)
      org.apache.kafka.connect.errors.ConnectException: Failed to parse and read a JSON value on settings JSON: The key offset in the JSON document is 42752 and is too big for the binary form of the document (34560)
      	at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:164)
      	at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:142)
      	at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:324)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:877)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:735)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:451)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient$5.run(BinaryLogClient.java:632)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to parse and read a JSON value on settings JSON: The key offset in the JSON document is 42752 and is too big for the binary form of the document (34560)
      	at io.debezium.connector.mysql.MySqlValueConverters.convertJson(MySqlValueConverters.java:209)
      	at io.debezium.connector.mysql.MySqlValueConverters.lambda$converter$30(MySqlValueConverters.java:117)
      	at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$170(TableSchemaBuilder.java:230)
      	at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:111)
      	at io.debezium.connector.mysql.RecordMakers$1.update(RecordMakers.java:232)
      	at io.debezium.connector.mysql.RecordMakers$RecordsForTable.update(RecordMakers.java:444)
      	at io.debezium.connector.mysql.BinlogReader.handleUpdate(BinlogReader.java:547)
      	at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:311)
      	... 5 more
      Caused by: java.io.IOException: The key offset in the JSON document is 42752 and is too big for the binary form of the document (34560)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.readUnsignedIndex(JsonBinary.java:867)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseObject(JsonBinary.java:334)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:216)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseObject(JsonBinary.java:398)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:216)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseObject(JsonBinary.java:398)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:216)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:210)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:181)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseAsString(JsonBinary.java:168)
      	at io.debezium.connector.mysql.MySqlValueConverters.convertJson(MySqlValueConverters.java:206)
      	... 12 more
      [2017-02-08 21:17:39,945] ERROR Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:143)
      [2017-02-08 21:17:39,944] ERROR Task debizium-connector-llama-0 threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask)
      org.apache.kafka.connect.errors.ConnectException: Failed to parse and read a JSON value on settings JSON: The key offset in the JSON document is 42752 and is too big for the binary form of the document (34560)
      	at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:164)
      	at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:142)
      	at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:324)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:877)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:735)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:451)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient$5.run(BinaryLogClient.java:632)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to parse and read a JSON value on settings JSON: The key offset in the JSON document is 42752 and is too big for the binary form of the document (34560)
      	at io.debezium.connector.mysql.MySqlValueConverters.convertJson(MySqlValueConverters.java:209)
      	at io.debezium.connector.mysql.MySqlValueConverters.lambda$converter$30(MySqlValueConverters.java:117)
      	at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$170(TableSchemaBuilder.java:230)
      	at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:111)
      	at io.debezium.connector.mysql.RecordMakers$1.update(RecordMakers.java:232)
      	at io.debezium.connector.mysql.RecordMakers$RecordsForTable.update(RecordMakers.java:444)
      	at io.debezium.connector.mysql.BinlogReader.handleUpdate(BinlogReader.java:547)
      	at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:311)
      	... 5 more
      Caused by: java.io.IOException: The key offset in the JSON document is 42752 and is too big for the binary form of the document (34560)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.readUnsignedIndex(JsonBinary.java:867)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseObject(JsonBinary.java:334)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:216)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseObject(JsonBinary.java:398)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:216)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseObject(JsonBinary.java:398)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:216)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:210)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parse(JsonBinary.java:181)
      	at com.github.shyiko.mysql.binlog.event.deserialization.json.JsonBinary.parseAsString(JsonBinary.java:168)
      	at io.debezium.connector.mysql.MySqlValueConverters.convertJson(MySqlValueConverters.java:206)
      	... 12 more
      
      

              rhauch Randall Hauch (Inactive)
              akshathpatkar Akshath Patkar (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: