Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6573

Refactor errors.max.retries to common connector framework

XMLWordPrintable

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      when mysql on aws(rds) is changed by blue/green update(binlog become invalid), mysql connector keep trying to connect mysql db. and it's status is running 

      the bin log is not valid, so the connector must be failed

       

      here is error log and it try to connect repeatedly

      [2023-06-19 06:59:55,690] ERROR Producer failure (io.debezium.pipeline.ErrorHandler)
      io.debezium.DebeziumException: Client requested master to start replication from position > file size Error code: 1236; SQLSTATE: HY000.
        at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1236)
        at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1281)
        at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079)
        at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
        at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
        at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: com.github.shyiko.mysql.binlog.network.ServerException: Client requested master to start replication from position > file size
        at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1043)
        ... 3 more
      [2023-06-19 06:59:55,691] WARN Retry 1 of unlimited retries will be attempted (io.debezium.pipeline.ErrorHandler)
      [2023-06-19 06:59:55,692] INFO Stopped reading binlog after 0 events, no new offset was recorded (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
      [2023-06-19 06:59:55,969] WARN Going to restart connector after 10 sec. after a retriable exception (io.debezium.connector.common.BaseSourceTask)
      [2023-06-19 07:00:00,677] INFO Finished streaming (io.debezium.pipeline.ChangeEventSourceCoordinator)
      [2023-06-19 07:00:00,680] INFO Connection gracefully closed (io.debezium.jdbc.JdbcConnection)
      [2023-06-19 07:00:00,681] INFO [Producer clientId=_mysql.ver.up-schemahistory] Closing the Kafka producer with timeoutMillis = 30000 ms. (org.apache.kafka.clients.producer.KafkaProducer)
      [2023-06-19 07:00:00,684] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
      [2023-06-19 07:00:00,684] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)
      [2023-06-19 07:00:00,684] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
      [2023-06-19 07:00:00,684] INFO App info kafka.producer for _mysql.ver.up-schemahistory unregistered (org.apache.kafka.common.utils.AppInfoParser)
      [2023-06-19 07:00:00,684] WARN WorkerSourceTask{id=source.mysql.ver.up-0} failed to poll records from SourceTask. Will retry operation. (org.apache.kafka.connect.runtime.WorkerSourceTask)
      org.apache.kafka.connect.errors.RetriableException: An exception occurred in the change event producer. This connector will be restarted.
        at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:68)
        at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1281)
        at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079)
        at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
        at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
        at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: io.debezium.DebeziumException: Client requested master to start replication from position > file size Error code: 1236; SQLSTATE: HY000.
        at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1236)
        ... 5 more
      Caused by: com.github.shyiko.mysql.binlog.network.ServerException: Client requested master to start replication from position > file size
        at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1043)
        ... 3 more
      [2023-06-19 07:00:00,685] INFO Awaiting end of restart backoff period after a retriable error (io.debezium.connector.common.BaseSourceTask)
      [2023-06-19 07:00:02,685] INFO Awaiting end of restart backoff period after a retriable error (io.debezium.connector.common.BaseSourceTask)
      [2023-06-19 07:00:04,685] INFO Awaiting end of restart backoff period after a retriable error (io.debezium.connector.common.BaseSourceTask)
      [2023-06-19 07:00:06,686] INFO Awaiting end of restart backoff period after a retriable error (io.debezium.connector.common.BaseSourceTask)
      [2023-06-19 07:00:08,686] INFO Awaiting end of restart backoff period after a retriable error (io.debezium.connector.common.BaseSourceTask)

      What Debezium connector do you use and what version?

      2.2.1

      Implementation ideas (optional)

      add max retries to mysql connector config like sqlserver config

              Unassigned Unassigned
              jaegwon.seo wornjs seo (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: