Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3722

In MySQL,When the network is down and back up,the Debezium Engine doesn't work well.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Major Major
    • None
    • 1.6.0.Final
    • mysql-connector
    • False
    • False
    • Undefined
    • Hide

      i use kafka,my code is:

      ...
      engine = DebeziumEngine.create(Json.class)
      .using(props)
      .using((success, message, error)->{
      completionCallback(success, message, error);
      })
      .notifying(record -> {
      notifying(record);
      }).build();
      // Run the engine asynchronously ...
      ExecutorService executor = Executors.newSingleThreadExecutor();
      executor.execute(engine);
      ...
      

      the network is down,the exception is:

      2021-07-09 08:05:37.968 ERROR 13460 --- [30.233.147:3306] i.d.c.m.MySqlStreamingChangeEventSource  : Error during binlog processing. Last offset stored = null, binlog reader near position = mysql-bin.000898/2027276352021-07-09 08:05:37.968 ERROR 13460 --- [30.233.147:3306] i.d.c.m.MySqlStreamingChangeEventSource  : Error during binlog processing. Last offset stored = null, binlog reader near position = mysql-bin.000898/2027276352021-07-09 08:05:37.973  INFO 13460 --- [d - cdc_offsets] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer-null-1, groupId=null] Error sending fetch request (sessionId=247403010, epoch=120) to node 0:
      org.apache.kafka.common.errors.DisconnectException: null
      2021-07-09 08:05:37.974 ERROR 13460 --- [30.233.147:3306] io.debezium.pipeline.ErrorHandler        : Producer failure
      io.debezium.DebeziumException: Network is unreachable: recv failed at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1154) ~[debezium-connector-mysql-1.6.0.Final.jar:1.6.0.Final] at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1199) [debezium-connector-mysql-1.6.0.Final.jar:1.6.0.Final] at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:973) [mysql-binlog-connector-java-0.25.1.jar:0.25.1] at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:606) [mysql-binlog-connector-java-0.25.1.jar:0.25.1] at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:850) [mysql-binlog-connector-java-0.25.1.jar:0.25.1] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_271]Caused by: java.net.SocketException: Network is unreachable: recv failed at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.8.0_271] at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[na:1.8.0_271] at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[na:1.8.0_271] at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[na:1.8.0_271] at com.github.shyiko.mysql.binlog.io.BufferedSocketInputStream.read(BufferedSocketInputStream.java:51) ~[mysql-binlog-connector-java-0.25.1.jar:0.25.1] at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readWithinBlockBoundaries(ByteArrayInputStream.java:222) ~[mysql-binlog-connector-java-0.25.1.jar:0.25.1] at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.peek(ByteArrayInputStream.java:194) ~[mysql-binlog-connector-java-0.25.1.jar:0.25.1] at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:930) [mysql-binlog-connector-java-0.25.1.jar:0.25.1] ... 3 common frames omitted
      2021-07-09 08:05:37.975  INFO 13460 --- [30.233.147:3306] i.d.c.m.MySqlStreamingChangeEventSource  : Stopped reading binlog after 0 events, no new offset was recorded2021-07-09 08:05:38.158  INFO 13460 --- [pool-1-thread-1] i.d.connector.common.BaseSourceTask      : Stopping down connector2021-07-09 08:05:38.202  INFO 13460 --- [rce-coordinator] i.d.p.ChangeEventSourceCoordinator       : Finished streaming2021-07-09 08:05:38.237 ERROR 13460 --- [pool-1-thread-1] i.d.connector.mysql.MySqlConnectorTask   : Exception while closing JDBC connection
      java.sql.SQLNonTransientConnectionException: Communications link failure during rollback(). Transaction resolution unknown. at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110) ~[mysql-connector-java-8.0.25.jar:8.0.25] at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) ~[mysql-connector-java-8.0.25.jar:8.0.25] at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89) ~[mysql-connector-java-8.0.25.jar:8.0.25] at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63) ~[mysql-connector-java-8.0.25.jar:8.0.25] at com.mysql.cj.jdbc.ConnectionImpl.rollback(ConnectionImpl.java:1853) ~[mysql-connector-java-8.0.25.jar:8.0.25] at com.mysql.cj.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:1710) ~[mysql-connector-java-8.0.25.jar:8.0.25] at com.mysql.cj.jdbc.ConnectionImpl.close(ConnectionImpl.java:718) ~[mysql-connector-java-8.0.25.jar:8.0.25] at io.debezium.jdbc.JdbcConnection.lambda$doClose$3(JdbcConnection.java:950) ~[debezium-core-1.6.0.Final.jar:1.6.0.Final] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_271] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_271] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_271] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_271]
      2021-07-09 08:05:38.238  INFO 13460 --- [pool-1-thread-1] o.a.k.clients.producer.KafkaProducer     : [Producer clientId=device-connector-dbhistory] Closing the Kafka producer with timeoutMillis = 30000 ms.2021-07-09 08:05:38.239  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Metrics scheduler closed2021-07-09 08:05:38.239  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Closing reporter org.apache.kafka.common.metrics.JmxReporter2021-07-09 08:05:38.239  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Metrics reporters closed2021-07-09 08:05:38.239  INFO 13460 --- [pool-1-thread-1] o.a.kafka.common.utils.AppInfoParser     : App info kafka.producer for device-connector-dbhistory unregistered2021-07-09 08:05:48.221  INFO 13460 --- [d - cdc_offsets] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer-null-1, groupId=null] Error sending fetch request (sessionId=247403010, epoch=INITIAL) to node 0:
      org.apache.kafka.common.errors.DisconnectException: null
      2021-07-09 08:05:59.074  WARN 13460 --- [| adminclient-2] org.apache.kafka.clients.NetworkClient   : [AdminClient clientId=adminclient-2] Connection to node 0 (/172.30.127.87:9092) could not be established. Broker may not be available.2021-07-09 08:06:07.971  INFO 13460 --- [| adminclient-2] o.a.k.c.a.i.AdminMetadataManager         : [AdminClient clientId=adminclient-2] Metadata update failed
      org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1625789167970, tries=2, nextAllowedTryMs=1625789168071) timed out at 1625789167971 after 2 attempt(s)Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata
      2021-07-09 08:06:08.465  INFO 13460 --- [d - cdc_offsets] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer-null-1, groupId=null] Error sending fetch request (sessionId=247403010, epoch=INITIAL) to node 0:
      org.apache.kafka.common.errors.DisconnectException: null
      2021-07-09 08:06:20.178  WARN 13460 --- [| adminclient-2] org.apache.kafka.clients.NetworkClient   : [AdminClient clientId=adminclient-2] Connection to node 0 (/172.30.127.87:9092) could not be established. Broker may not be available.2021-07-09 08:06:29.547  WARN 13460 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient   : [Producer clientId=producer-1] Connection to node 0 (/172.30.127.87:9092) could not be established. Broker may not be available.2021-07-09 08:06:29.670  WARN 13460 --- [d - cdc_offsets] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-null-1, groupId=null] Connection to node 0 (/172.30.127.87:9092) could not be established. Broker may not be available.2021-07-09 08:06:29.670  INFO 13460 --- [d - cdc_offsets] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer-null-1, groupId=null] Error sending fetch request (sessionId=247403010, epoch=INITIAL) to node 0:
      org.apache.kafka.common.errors.DisconnectException: null
      2021-07-09 08:06:38.243 ERROR 13460 --- [pool-1-thread-1] io.debezium.embedded.EmbeddedEngine      : Failed to flush EmbeddedEngine\{id=engine} offsets to storage: 
      org.apache.kafka.common.errors.TimeoutException: Topic cdc_offsets not present in metadata after 60000 ms.
      2021-07-09 08:06:38.244 ERROR 13460 --- [pool-1-thread-1] io.debezium.embedded.EmbeddedEngine      : Flush of EmbeddedEngine\{id=engine} offsets threw an unexpected exception: 
      java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Topic cdc_offsets not present in metadata after 60000 ms. at org.apache.kafka.connect.storage.KafkaOffsetBackingStore$SetCallbackFuture.get(KafkaOffsetBackingStore.java:270) ~[connect-runtime-2.7.1.jar:na] at org.apache.kafka.connect.storage.KafkaOffsetBackingStore$SetCallbackFuture.get(KafkaOffsetBackingStore.java:203) ~[connect-runtime-2.7.1.jar:na] at io.debezium.embedded.EmbeddedEngine.commitOffsets(EmbeddedEngine.java:1001) [debezium-embedded-1.6.0.Final.jar:1.6.0.Final] at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:850) [debezium-embedded-1.6.0.Final.jar:1.6.0.Final] at io.debezium.embedded.ConvertingEngineBuilder$2.run(ConvertingEngineBuilder.java:188) [debezium-embedded-1.6.0.Final.jar:1.6.0.Final] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_271] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_271] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_271]Caused by: org.apache.kafka.common.errors.TimeoutException: Topic cdc_offsets not present in metadata after 60000 ms.
      2021-07-09 08:06:38.244  INFO 13460 --- [pool-1-thread-1] o.a.k.c.storage.KafkaOffsetBackingStore  : Stopping KafkaOffsetBackingStore2021-07-09 08:06:38.244  INFO 13460 --- [pool-1-thread-1] o.a.kafka.connect.util.KafkaBasedLog     : Stopping KafkaBasedLog for topic cdc_offsets2021-07-09 08:06:38.244  INFO 13460 --- [pool-1-thread-1] o.a.k.clients.producer.KafkaProducer     : [Producer clientId=producer-1] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.2021-07-09 08:06:38.245  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Metrics scheduler closed2021-07-09 08:06:38.245  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Closing reporter org.apache.kafka.common.metrics.JmxReporter2021-07-09 08:06:38.246  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Metrics reporters closed2021-07-09 08:06:38.246  INFO 13460 --- [pool-1-thread-1] o.a.kafka.common.utils.AppInfoParser     : App info kafka.producer for producer-1 unregistered2021-07-09 08:06:38.246  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Metrics scheduler closed2021-07-09 08:06:38.246  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Closing reporter org.apache.kafka.common.metrics.JmxReporter2021-07-09 08:06:38.246  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Metrics reporters closed2021-07-09 08:06:38.247  INFO 13460 --- [pool-1-thread-1] o.a.kafka.common.utils.AppInfoParser     : App info kafka.consumer for consumer-null-1 unregistered2021-07-09 08:06:38.247  INFO 13460 --- [pool-1-thread-1] o.a.kafka.connect.util.KafkaBasedLog     : Stopped KafkaBasedLog for topic cdc_offsets2021-07-09 08:06:38.248  INFO 13460 --- [| adminclient-2] o.a.kafka.common.utils.AppInfoParser     : App info kafka.admin.client for adminclient-2 unregistered2021-07-09 08:06:38.248  INFO 13460 --- [| adminclient-2] o.a.k.c.a.i.AdminMetadataManager         : [AdminClient clientId=adminclient-2] Metadata update failed
      org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1625789210178, tries=1, nextAllowedTryMs=-9223372036854775709) timed out at 9223372036854775807 after 1 attempt(s)Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata
      2021-07-09 08:06:38.249  INFO 13460 --- [| adminclient-2] org.apache.kafka.common.metrics.Metrics  : Metrics scheduler closed2021-07-09 08:06:38.249  INFO 13460 --- [| adminclient-2] org.apache.kafka.common.metrics.Metrics  : Closing reporter org.apache.kafka.common.metrics.JmxReporter2021-07-09 08:06:38.249  INFO 13460 --- [| adminclient-2] org.apache.kafka.common.metrics.Metrics  : Metrics reporters closed2021-07-09 08:06:38.250  INFO 13460 --- [pool-1-thread-1] o.a.k.c.storage.KafkaOffsetBackingStore  : Stopped KafkaOffsetBackingStore========completionCallback==========Error while trying to run connector class 'io.debezium.connector.mysql.MySqlConnector'org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
      
      Show
      i use kafka,my code is: ... engine = DebeziumEngine.create(Json.class) .using(props) .using((success, message, error)->{ completionCallback(success, message, error); }) .notifying(record -> { notifying(record); }).build(); // Run the engine asynchronously ... ExecutorService executor = Executors.newSingleThreadExecutor(); executor.execute(engine); ... the network is down,the exception is: 2021-07-09 08:05:37.968 ERROR 13460 --- [30.233.147:3306] i.d.c.m.MySqlStreamingChangeEventSource  : Error during binlog processing. Last offset stored = null , binlog reader near position = mysql-bin.000898/2027276352021-07-09 08:05:37.968 ERROR 13460 --- [30.233.147:3306] i.d.c.m.MySqlStreamingChangeEventSource  : Error during binlog processing. Last offset stored = null , binlog reader near position = mysql-bin.000898/2027276352021-07-09 08:05:37.973  INFO 13460 --- [d - cdc_offsets] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer- null -1, groupId= null ] Error sending fetch request (sessionId=247403010, epoch=120) to node 0: org.apache.kafka.common.errors.DisconnectException: null 2021-07-09 08:05:37.974 ERROR 13460 --- [30.233.147:3306] io.debezium.pipeline.ErrorHandler        : Producer failure io.debezium.DebeziumException: Network is unreachable: recv failed at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1154) ~[debezium-connector-mysql-1.6.0.Final.jar:1.6.0.Final] at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1199) [debezium-connector-mysql-1.6.0.Final.jar:1.6.0.Final] at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:973) [mysql-binlog-connector-java-0.25.1.jar:0.25.1] at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:606) [mysql-binlog-connector-java-0.25.1.jar:0.25.1] at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:850) [mysql-binlog-connector-java-0.25.1.jar:0.25.1] at java.lang. Thread .run( Thread .java:748) [na:1.8.0_271]Caused by: java.net.SocketException: Network is unreachable: recv failed at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.8.0_271] at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[na:1.8.0_271] at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[na:1.8.0_271] at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[na:1.8.0_271] at com.github.shyiko.mysql.binlog.io.BufferedSocketInputStream.read(BufferedSocketInputStream.java:51) ~[mysql-binlog-connector-java-0.25.1.jar:0.25.1] at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readWithinBlockBoundaries(ByteArrayInputStream.java:222) ~[mysql-binlog-connector-java-0.25.1.jar:0.25.1] at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.peek(ByteArrayInputStream.java:194) ~[mysql-binlog-connector-java-0.25.1.jar:0.25.1] at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:930) [mysql-binlog-connector-java-0.25.1.jar:0.25.1] ... 3 common frames omitted 2021-07-09 08:05:37.975  INFO 13460 --- [30.233.147:3306] i.d.c.m.MySqlStreamingChangeEventSource  : Stopped reading binlog after 0 events, no new offset was recorded2021-07-09 08:05:38.158  INFO 13460 --- [pool-1-thread-1] i.d.connector.common.BaseSourceTask      : Stopping down connector2021-07-09 08:05:38.202  INFO 13460 --- [rce-coordinator] i.d.p.ChangeEventSourceCoordinator       : Finished streaming2021-07-09 08:05:38.237 ERROR 13460 --- [pool-1-thread-1] i.d.connector.mysql.MySqlConnectorTask   : Exception while closing JDBC connection java.sql.SQLNonTransientConnectionException: Communications link failure during rollback(). Transaction resolution unknown. at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110) ~[mysql-connector-java-8.0.25.jar:8.0.25] at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) ~[mysql-connector-java-8.0.25.jar:8.0.25] at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89) ~[mysql-connector-java-8.0.25.jar:8.0.25] at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63) ~[mysql-connector-java-8.0.25.jar:8.0.25] at com.mysql.cj.jdbc.ConnectionImpl.rollback(ConnectionImpl.java:1853) ~[mysql-connector-java-8.0.25.jar:8.0.25] at com.mysql.cj.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:1710) ~[mysql-connector-java-8.0.25.jar:8.0.25] at com.mysql.cj.jdbc.ConnectionImpl.close(ConnectionImpl.java:718) ~[mysql-connector-java-8.0.25.jar:8.0.25] at io.debezium.jdbc.JdbcConnection.lambda$doClose$3(JdbcConnection.java:950) ~[debezium-core-1.6.0.Final.jar:1.6.0.Final] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_271] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_271] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_271] at java.lang. Thread .run( Thread .java:748) [na:1.8.0_271] 2021-07-09 08:05:38.238  INFO 13460 --- [pool-1-thread-1] o.a.k.clients.producer.KafkaProducer     : [Producer clientId=device-connector-dbhistory] Closing the Kafka producer with timeoutMillis = 30000 ms.2021-07-09 08:05:38.239  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Metrics scheduler closed2021-07-09 08:05:38.239  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Closing reporter org.apache.kafka.common.metrics.JmxReporter2021-07-09 08:05:38.239  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Metrics reporters closed2021-07-09 08:05:38.239  INFO 13460 --- [pool-1-thread-1] o.a.kafka.common.utils.AppInfoParser     : App info kafka.producer for device-connector-dbhistory unregistered2021-07-09 08:05:48.221  INFO 13460 --- [d - cdc_offsets] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer- null -1, groupId= null ] Error sending fetch request (sessionId=247403010, epoch=INITIAL) to node 0: org.apache.kafka.common.errors.DisconnectException: null 2021-07-09 08:05:59.074  WARN 13460 --- [| adminclient-2] org.apache.kafka.clients.NetworkClient   : [AdminClient clientId=adminclient-2] Connection to node 0 (/172.30.127.87:9092) could not be established. Broker may not be available.2021-07-09 08:06:07.971  INFO 13460 --- [| adminclient-2] o.a.k.c.a.i.AdminMetadataManager         : [AdminClient clientId=adminclient-2] Metadata update failed org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1625789167970, tries=2, nextAllowedTryMs=1625789168071) timed out at 1625789167971 after 2 attempt(s)Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata 2021-07-09 08:06:08.465  INFO 13460 --- [d - cdc_offsets] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer- null -1, groupId= null ] Error sending fetch request (sessionId=247403010, epoch=INITIAL) to node 0: org.apache.kafka.common.errors.DisconnectException: null 2021-07-09 08:06:20.178  WARN 13460 --- [| adminclient-2] org.apache.kafka.clients.NetworkClient   : [AdminClient clientId=adminclient-2] Connection to node 0 (/172.30.127.87:9092) could not be established. Broker may not be available.2021-07-09 08:06:29.547  WARN 13460 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient   : [Producer clientId=producer-1] Connection to node 0 (/172.30.127.87:9092) could not be established. Broker may not be available.2021-07-09 08:06:29.670  WARN 13460 --- [d - cdc_offsets] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer- null -1, groupId= null ] Connection to node 0 (/172.30.127.87:9092) could not be established. Broker may not be available.2021-07-09 08:06:29.670  INFO 13460 --- [d - cdc_offsets] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer- null -1, groupId= null ] Error sending fetch request (sessionId=247403010, epoch=INITIAL) to node 0: org.apache.kafka.common.errors.DisconnectException: null 2021-07-09 08:06:38.243 ERROR 13460 --- [pool-1-thread-1] io.debezium.embedded.EmbeddedEngine      : Failed to flush EmbeddedEngine\{id=engine} offsets to storage:  org.apache.kafka.common.errors.TimeoutException: Topic cdc_offsets not present in metadata after 60000 ms. 2021-07-09 08:06:38.244 ERROR 13460 --- [pool-1-thread-1] io.debezium.embedded.EmbeddedEngine      : Flush of EmbeddedEngine\{id=engine} offsets threw an unexpected exception:  java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Topic cdc_offsets not present in metadata after 60000 ms. at org.apache.kafka.connect.storage.KafkaOffsetBackingStore$SetCallbackFuture.get(KafkaOffsetBackingStore.java:270) ~[connect-runtime-2.7.1.jar:na] at org.apache.kafka.connect.storage.KafkaOffsetBackingStore$SetCallbackFuture.get(KafkaOffsetBackingStore.java:203) ~[connect-runtime-2.7.1.jar:na] at io.debezium.embedded.EmbeddedEngine.commitOffsets(EmbeddedEngine.java:1001) [debezium-embedded-1.6.0.Final.jar:1.6.0.Final] at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:850) [debezium-embedded-1.6.0.Final.jar:1.6.0.Final] at io.debezium.embedded.ConvertingEngineBuilder$2.run(ConvertingEngineBuilder.java:188) [debezium-embedded-1.6.0.Final.jar:1.6.0.Final] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_271] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_271] at java.lang. Thread .run( Thread .java:748) [na:1.8.0_271]Caused by: org.apache.kafka.common.errors.TimeoutException: Topic cdc_offsets not present in metadata after 60000 ms. 2021-07-09 08:06:38.244  INFO 13460 --- [pool-1-thread-1] o.a.k.c.storage.KafkaOffsetBackingStore  : Stopping KafkaOffsetBackingStore2021-07-09 08:06:38.244  INFO 13460 --- [pool-1-thread-1] o.a.kafka.connect.util.KafkaBasedLog     : Stopping KafkaBasedLog for topic cdc_offsets2021-07-09 08:06:38.244  INFO 13460 --- [pool-1-thread-1] o.a.k.clients.producer.KafkaProducer     : [Producer clientId=producer-1] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.2021-07-09 08:06:38.245  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Metrics scheduler closed2021-07-09 08:06:38.245  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Closing reporter org.apache.kafka.common.metrics.JmxReporter2021-07-09 08:06:38.246  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Metrics reporters closed2021-07-09 08:06:38.246  INFO 13460 --- [pool-1-thread-1] o.a.kafka.common.utils.AppInfoParser     : App info kafka.producer for producer-1 unregistered2021-07-09 08:06:38.246  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Metrics scheduler closed2021-07-09 08:06:38.246  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Closing reporter org.apache.kafka.common.metrics.JmxReporter2021-07-09 08:06:38.246  INFO 13460 --- [pool-1-thread-1] org.apache.kafka.common.metrics.Metrics  : Metrics reporters closed2021-07-09 08:06:38.247  INFO 13460 --- [pool-1-thread-1] o.a.kafka.common.utils.AppInfoParser     : App info kafka.consumer for consumer- null -1 unregistered2021-07-09 08:06:38.247  INFO 13460 --- [pool-1-thread-1] o.a.kafka.connect.util.KafkaBasedLog     : Stopped KafkaBasedLog for topic cdc_offsets2021-07-09 08:06:38.248  INFO 13460 --- [| adminclient-2] o.a.kafka.common.utils.AppInfoParser     : App info kafka.admin.client for adminclient-2 unregistered2021-07-09 08:06:38.248  INFO 13460 --- [| adminclient-2] o.a.k.c.a.i.AdminMetadataManager         : [AdminClient clientId=adminclient-2] Metadata update failed org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1625789210178, tries=1, nextAllowedTryMs=-9223372036854775709) timed out at 9223372036854775807 after 1 attempt(s)Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata 2021-07-09 08:06:38.249  INFO 13460 --- [| adminclient-2] org.apache.kafka.common.metrics.Metrics  : Metrics scheduler closed2021-07-09 08:06:38.249  INFO 13460 --- [| adminclient-2] org.apache.kafka.common.metrics.Metrics  : Closing reporter org.apache.kafka.common.metrics.JmxReporter2021-07-09 08:06:38.249  INFO 13460 --- [| adminclient-2] org.apache.kafka.common.metrics.Metrics  : Metrics reporters closed2021-07-09 08:06:38.250  INFO 13460 --- [pool-1-thread-1] o.a.k.c.storage.KafkaOffsetBackingStore  : Stopped KafkaOffsetBackingStore========completionCallback==========Error while trying to run connector class 'io.debezium.connector.mysql.MySqlConnector' org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.

      Could you help improve the relevant support for MySQL?
      When the network is down and back up,the Debezium Engine doesn't work well.

       

            Unassigned Unassigned
            gopatter 王子 王 (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: