Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3833

Upgrade MySQL driver dependency to latest version

    XMLWordPrintable

Details

    Description

      When setting up a simple "Embedded Debezium" project a crash is caused if using a newer 8.x version of Mysql-connector java.

      Stack trace:

      java.lang.IllegalAccessError: class io.debezium.connector.mysql.antlr.MySqlAntlrDdlParser tried to access private field com.mysql.cj.CharsetMapping.COLLATION_INDEX_TO_COLLATION_NAME (io.debezium.connector.mysql.antlr.MySqlAntlrDdlParser and com.mysql.cj.CharsetMapping are in unnamed module of loader 'app')
      	at io.debezium.connector.mysql.antlr.MySqlAntlrDdlParser.extractCharset(MySqlAntlrDdlParser.java:403) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.antlr.listener.CreateAndAlterDatabaseParserListener.enterCreateDatabaseOption(CreateAndAlterDatabaseParserListener.java:49) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.ddl.parser.mysql.generated.MySqlParser$CreateDatabaseOptionContext.enterRule(MySqlParser.java:5906) ~[debezium-ddl-parser-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.antlr.ProxyParseTreeListenerUtil.delegateEnterRule(ProxyParseTreeListenerUtil.java:46) ~[debezium-ddl-parser-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.antlr.listener.MySqlAntlrDdlParserListener.enterEveryRule(MySqlAntlrDdlParserListener.java:89) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.enterRule(ParseTreeWalker.java:41) ~[antlr4-runtime-4.8.jar:4.8]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:25) ~[antlr4-runtime-4.8.jar:4.8]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[antlr4-runtime-4.8.jar:4.8]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[antlr4-runtime-4.8.jar:4.8]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[antlr4-runtime-4.8.jar:4.8]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[antlr4-runtime-4.8.jar:4.8]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[antlr4-runtime-4.8.jar:4.8]
      	at io.debezium.antlr.AntlrDdlParser.parse(AntlrDdlParser.java:87) ~[debezium-ddl-parser-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.MySqlDatabaseSchema.parseDdl(MySqlDatabaseSchema.java:213) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.MySqlDatabaseSchema.parseSnapshotDdl(MySqlDatabaseSchema.java:195) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.MySqlSnapshotChangeEventSource.addSchemaEvent(MySqlSnapshotChangeEventSource.java:293) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.MySqlSnapshotChangeEventSource.readTableStructure(MySqlSnapshotChangeEventSource.java:346) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.MySqlSnapshotChangeEventSource.readTableStructure(MySqlSnapshotChangeEventSource.java:47) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:121) ~[debezium-core-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:70) ~[debezium-core-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:125) ~[debezium-core-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
      	at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) ~[na:na]
      	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) ~[na:na]
      	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
      	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
      	at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
      
      2021-08-08 01:38:59.520  INFO 1373053 --- [pool-1-thread-1] i.d.connector.common.BaseSourceTask      : Stopping down connector
      2021-08-08 01:38:59.522  INFO 1373053 --- [pool-3-thread-1] io.debezium.jdbc.JdbcConnection          : Connection gracefully closed
      2021-08-08 01:38:59.523  INFO 1373053 --- [pool-1-thread-1] o.a.k.c.storage.FileOffsetBackingStore   : Stopped FileOffsetBackingStore
      2021-08-08 01:38:59.524 ERROR 1373053 --- [pool-1-thread-1] io.debezium.embedded.EmbeddedEngine      : Error while trying to run connector class 'io.debezium.connector.mysql.MySqlConnector'
      
      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
      	at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42) ~[debezium-core-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:142) ~[debezium-core-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
      	at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) ~[na:na]
      	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) ~[na:na]
      	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
      	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
      	at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
      Caused by: java.lang.IllegalAccessError: class io.debezium.connector.mysql.antlr.MySqlAntlrDdlParser tried to access private field com.mysql.cj.CharsetMapping.COLLATION_INDEX_TO_COLLATION_NAME (io.debezium.connector.mysql.antlr.MySqlAntlrDdlParser and com.mysql.cj.CharsetMapping are in unnamed module of loader 'app')
      	at io.debezium.connector.mysql.antlr.MySqlAntlrDdlParser.extractCharset(MySqlAntlrDdlParser.java:403) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.antlr.listener.CreateAndAlterDatabaseParserListener.enterCreateDatabaseOption(CreateAndAlterDatabaseParserListener.java:49) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.ddl.parser.mysql.generated.MySqlParser$CreateDatabaseOptionContext.enterRule(MySqlParser.java:5906) ~[debezium-ddl-parser-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.antlr.ProxyParseTreeListenerUtil.delegateEnterRule(ProxyParseTreeListenerUtil.java:46) ~[debezium-ddl-parser-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.antlr.listener.MySqlAntlrDdlParserListener.enterEveryRule(MySqlAntlrDdlParserListener.java:89) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.enterRule(ParseTreeWalker.java:41) ~[antlr4-runtime-4.8.jar:4.8]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:25) ~[antlr4-runtime-4.8.jar:4.8]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[antlr4-runtime-4.8.jar:4.8]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[antlr4-runtime-4.8.jar:4.8]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[antlr4-runtime-4.8.jar:4.8]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[antlr4-runtime-4.8.jar:4.8]
      	at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[antlr4-runtime-4.8.jar:4.8]
      	at io.debezium.antlr.AntlrDdlParser.parse(AntlrDdlParser.java:87) ~[debezium-ddl-parser-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.MySqlDatabaseSchema.parseDdl(MySqlDatabaseSchema.java:213) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.MySqlDatabaseSchema.parseSnapshotDdl(MySqlDatabaseSchema.java:195) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.MySqlSnapshotChangeEventSource.addSchemaEvent(MySqlSnapshotChangeEventSource.java:293) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.MySqlSnapshotChangeEventSource.readTableStructure(MySqlSnapshotChangeEventSource.java:346) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.connector.mysql.MySqlSnapshotChangeEventSource.readTableStructure(MySqlSnapshotChangeEventSource.java:47) ~[debezium-connector-mysql-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:121) ~[debezium-core-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:70) ~[debezium-core-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:125) ~[debezium-core-1.7.0.Alpha1.jar:1.7.0.Alpha1]
      	... 6 common frames omitted
      
      

      Cause is that Debezium MySql connector accesses a static field from Mysql Java connector directly from: MySqlAntlrDdlParser.java the CharsetMapping.COLLATION_INDEX_TO_COLLATION_NAME static field.

      This field was set to private in a recent patch to MySQL java connector (Commit with the change)

      Can attempt to fix this in a PR if requested, but could not find good overview on if this version should be supported, though I would expect compatability with all 8.0.x branch.
       

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              chargedpeptide Erik Malm (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: