Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5840

Can't start KAfka Connector after upgrade Debezium MySQL plugin from 1.3 to 2.0

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • Icon: Major Major
    • None
    • None
    • None
    • None

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      We use the MySQL Debezium 1.3

      What is the connector configuration?

       

      What is the captured database version and mode of depoyment?

      (E.g. on-premises, with a specific cloud provider, etc.)

      we use MySQL 8.x

      What behaviour do you expect?

      With the Debezium MySQL connector 1.3 we set in the KAfka Connect service the path to debezium:

      plugin.path=/var/lib/kafka/debezium

       

      In that directory we have the subdirectory: debezium-connector-mysql

      Inn that diretcory (debezium-connector-mysql) you have the verison 1.3 

      rw-rr- 1 kafka confluent  337904 Nov 15 16:48 antlr4-runtime-4.7.2.jar
      rw-rr- 1 kafka confluent  205710 Nov 15 16:48 CHANGELOG.md
      rw-rr- 1 kafka confluent   19372 Nov 15 16:48 CONTRIBUTE.md
      rw-rr- 1 kafka confluent    3364 Nov 15 16:48 COPYRIGHT.txt
      rw-rr- 1 kafka confluent   20270 Nov 15 16:48 debezium-api-1.3.1.Final.jar
      rw-rr- 1 kafka confluent  259403 Nov 15 16:48 debezium-connector-mysql-1.3.1.Final.jar
      rw-rr- 1 kafka confluent  839613 Nov 15 16:48 debezium-core-1.3.1.Final.jar
      rw-rr- 1 kafka confluent 2722938 Nov 15 16:48 debezium-ddl-parser-1.3.1.Final.jar
      rw-rr- 1 kafka confluent  129157 Nov 15 16:48 LICENSE-3rd-PARTIES.txt
      rw-rr- 1 kafka confluent   11357 Nov 15 16:48 LICENSE.txt
      rw-rr- 1 kafka confluent  182602 Nov 15 16:48 mysql-binlog-connector-java-0.23.1.jar
      rw-rr- 1 kafka confluent 2356711 Nov 15 16:48 mysql-connector-java-8.0.19.jar
      rw-rr- 1 kafka confluent   13825 Nov 15 16:48 README.md

       

      And we can start the Connector; it founds the lib of Deebzium MySQL.

      [2022-11-15 16:48:54,602] INFO Scanning for plugin classes. This might take a moment ... (org.apache.kafka.connect.cli.ConnectDistributed)
      [2022-11-15 16:48:54,617] INFO Loading plugin from: /var/lib/kafka/debezium/debezium-connector-mysql (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:48:54,970] INFO Registered loader: PluginClassLoader{pluginLocation=file:/var/lib/kafka/debezium/debezium-connector-mysql/} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:48:54,970] INFO Added plugin 'io.debezium.connector.mysql.MySqlConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:48:54,970] INFO Added plugin 'io.debezium.converters.ByteBufferConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:48:54,970] INFO Added plugin 'io.debezium.converters.CloudEventsConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:48:54,970] INFO Added plugin 'io.debezium.transforms.outbox.EventRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:48:54,970] INFO Added plugin 'io.debezium.transforms.ExtractNewRecordState' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:48:54,970] INFO Added plugin 'io.debezium.transforms.ByLogicalTableRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:48:54,970] INFO Added plugin 'org.apache.kafka.connect.connector.policy.AllConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:48:54,970] INFO Added plugin 'org.apache.kafka.connect.connector.policy.PrincipalConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:48:54,970] INFO Added plugin 'org.apache.kafka.connect.connector.policy.NoneConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)

       

       

      But when we replace the content of that directory with a new version of Debezium SQL : 2.0

      rw-rr- 1 kafka confluent  337864 Oct 14 12:26 antlr4-runtime-4.8.jar
      rw-rr- 1 kafka confluent  408916 Oct 14 13:26 CHANGELOG.md
      rw-rr- 1 kafka confluent   19246 Oct 14 13:26 CONTRIBUTE.md
      rw-rr- 1 kafka confluent    6489 Oct 14 13:26 COPYRIGHT.txt
      rw-rr- 1 kafka confluent   24404 Oct 14 13:26 debezium-api-2.0.0.Final.jar
      rw-rr- 1 kafka confluent  274264 Oct 14 13:28 debezium-connector-mysql-2.0.0.Final.jar
      rw-rr- 1 kafka confluent  970005 Oct 14 13:27 debezium-core-2.0.0.Final.jar
      rw-rr- 1 kafka confluent 3036392 Oct 14 13:27 debezium-ddl-parser-2.0.0.Final.jar
      rw-rr- 1 kafka confluent    7357 Oct 14 13:28 debezium-storage-file-2.0.0.Final.jar
      rw-rr- 1 kafka confluent   16141 Oct 14 13:28 debezium-storage-kafka-2.0.0.Final.jar
      rw-rr- 1 kafka confluent  129157 Oct 14 13:26 LICENSE-3rd-PARTIES.txt
      rw-rr- 1 kafka confluent   11357 Oct 14 13:26 LICENSE.txt
      rw-rr- 1 kafka confluent  209508 Oct 14 12:28 mysql-binlog-connector-java-0.27.2.jar
      rw-rr- 1 kafka confluent 2519547 Oct 14 12:28 mysql-connector-java-8.0.29.jar
      rw-rr- 1 kafka confluent   42582 Oct 14 13:28 mysql.json
      rw-rr- 1 kafka confluent   19608 Oct 14 13:26 README_JA.md
      rw-rr- 1 kafka confluent   18365 Oct 14 13:26 README_KO.md
      rw-rr- 1 kafka confluent   16594 Oct 14 13:26 README.md
      rw-rr- 1 kafka confluent   15739 Oct 14 13:26 README_ZH.md
      rw-rr- 1 kafka confluent 6762522 Oct 14 12:28 zstd-jni-1.5.0-2.jar

       

       

       

      it complains that it doesn't found a lib

       and we don't see in the log the load of the different lib:
      [2022-11-15 16:45:29,661] INFO Loading plugin from: /var/lib/kafka/debezium/debezium-connector-mysql (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:45:30,046] INFO Registered loader: PluginClassLoader{pluginLocation=file:/var/lib/kafka/debezium/debezium-connector-mysql/} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:45:30,046] INFO Added plugin 'org.apache.kafka.connect.connector.policy.AllConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:45:30,046] INFO Added plugin 'org.apache.kafka.connect.connector.policy.PrincipalConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:45:30,046] INFO Added plugin 'org.apache.kafka.connect.connector.policy.NoneConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:45:32,526] INFO Registered loader: sun.misc.Launcher$AppClassLoader@764c12b6 (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:45:32,526] INFO Added plugin 'org.apache.kafka.connect.mirror.MirrorSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:45:32,526] INFO Added plugin 'org.apache.kafka.connect.file.FileStreamSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
      [2022-11-15 16:45:32,526] INFO Added plugin 'org.apache.kafka.connect.tools.MockSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)

       

      Then for each connector create we see the following: -->  (org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches io.debezium.connector.mysql.MySqlConnector, available connectors are: ) 

       

      [2022-11-15 16:45:33,640] ERROR Failed to start connector z05-02_dev (org.apache.kafka.connect.runtime.Worker)
      org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches io.debezium.connector.mysql.MySqlConnector, available connectors are: PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSinkConnector, name='org.apache.kafka.connect.file.FileStreamSinkConnector', version='6.1.1-ccs', encodedVersion=6.1.1-ccs, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSourceConnector, name='org.apache.kafka.connect.file.FileStreamSourceConnector', version='6.1.1-ccs', encodedVersion=6.1.1-ccs, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorCheckpointConnector, name='org.apache.kafka.connect.mirror.MirrorCheckpointConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorHeartbeatConnector, name='org.apache.kafka.connect.mirror.MirrorHeartbeatConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorSourceConnector, name='org.apache.kafka.connect.mirror.MirrorSourceConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockConnector, name='org.apache.kafka.connect.tools.MockConnector', version='6.1.1-ccs', encodedVersion=6.1.1-ccs, type=connector, typeName='connector', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.apache.kafka.connect.tools.MockSinkConnector', version='6.1.1-ccs', encodedVersion=6.1.1-ccs, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tools.MockSourceConnector', version='6.1.1-ccs', encodedVersion=6.1.1-ccs, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='6.1.1-ccs', encodedVersion=6.1.1-ccs, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect.tools.VerifiableSinkConnector', version='6.1.1-ccs', encodedVersion=6.1.1-ccs, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='6.1.1-ccs', encodedVersion=6.1.1-ccs, type=source, typeName='source', location='classpath'}
              at org.apache.kafka.connect.runtime.isolation.Plugins.connectorClass(Plugins.java:208)
              at org.apache.kafka.connect.runtime.isolation.Plugins.newConnector(Plugins.java:180)
              at org.apache.kafka.connect.runtime.Worker.startConnector(Worker.java:275)
              at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startConnector(DistributedHerder.java:1360)
              at org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1400(DistributedHerder.java:130)
              at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:1368)
              at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:1364)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:750)

       

       

      What need to change/set to make the 2.0 Debezium MySQL running  ?

       

      Regards,

      Eric

            Unassigned Unassigned
            eric.franckx@ngahr.com Eric FRANCKX (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: