Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5775

Read-Only implementation incremental snapshot does not work on MariaDB

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Major Major
    • None
    • None
    • mysql-connector
    • None
    • False
    • None
    • False

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      debezium-connector-mysql 1.9.6

      What is the connector configuration?

      {
          "connector.class": "io.debezium.connector.mysql.MySqlConnector",
          "database.history.consumer.sasl.client.callback.handler.class": "software.amazon.msk.auth.iam.IAMClientCallbackHandler",
          "database.history.consumer.sasl.jaas.config": "software.amazon.msk.auth.iam.IAMLoginModule required;",
          "database.history.consumer.sasl.mechanism": "AWS_MSK_IAM",
          "database.history.consumer.security.protocol": "SASL_SSL",
          "database.history.kafka.topic": "schema-changes.<topic>",
          "database.history.producer.sasl.client.callback.handler.class": "software.amazon.msk.auth.iam.IAMClientCallbackHandler",
          "database.history.producer.sasl.jaas.config": "software.amazon.msk.auth.iam.IAMLoginModule required;",
          "database.history.producer.sasl.mechanism": "AWS_MSK_IAM",
          "database.history.producer.security.protocol": "SASL_SSL",
          "database.history.sasl.client.callback.handler.class": "software.amazon.msk.auth.iam.IAMClientCallbackHandler",
          "database.history.sasl.jaas.config": "software.amazon.msk.auth.iam.IAMLoginModule required;",
          "database.history.sasl.mechanism": "AWS_MSK_IAM",
          "database.history.security.protocol": "SASL_SSL",
          "database.hostname": "<hostname>",
          "database.include.list": "<database>",
          "database.password": "<password>",
          "database.port": "<port>",
          "database.server.name": "<servername>",
          "database.user": "<username>",
          "include.schema.changes": "false",
          "key.converter.schemas.enable": "false",
          "key.converter": "org.apache.kafka.connect.json.JsonConverter",
          "table.include.list": "<tables>",
          "tasks.max": "1",
          "transforms": "topicName",
          "transforms.topicName.regex": "(.*)\\.(.*)\\.(.*)",
          "transforms.topicName.replacement": "<my-prefix>.$3",
          "transforms.topicName.type": "org.apache.kafka.connect.transforms.RegexRouter",
          "value.converter.schemas.enable": "false",
          "value.converter": "org.apache.kafka.connect.json.JsonConverter",
          "topic.creation.enable": "false",
          "topic.creation.default.replication.factor": "-1",
          "topic.creation.default.partitions": "-1",     
          "signal.kafka.bootstrap.servers": "<broker>",
          "signal.kafka.topic": "<signal-topic>",
          "read.only": "true"
      } 

      What is the captured database version and mode of depoyment?

      (E.g. on-premises, with a specific cloud provider, etc.)

      Amazon RDS mariadb 10.5.17 

      What behaviour do you expect?

      MariaDB has GTID enabled by default so I'd expect this GTID check to go through on mariadb. 

      What behaviour do you see?

      ERROR [connector-thread|task-0] Producer failure (io.debezium.pipeline.ErrorHandler:35) java.lang.UnsupportedOperationException: Read only connection requires GTID_MODE to be ON 

      This check condition in code looks wrong for mariadb as GTID_MODE variable is unsupported.

      ERROR 1193 (HY000): Unknown system variable 'GTID_MODE' 

      Do you see the same behaviour using the latest relesead Debezium version?

      (Ideally, also verify with latest Alpha/Beta/CR version)

      The latest Debezium Version v2.0.0 is still doing the same SQL statement check that's leading to this issue.

      Do you have the connector logs, ideally from start till finish?

      (You might be asked later to provide DEBUG/TRACE level log)

      Worker-0e19902a182ab6160] [2022-10-27 12:01:25,651] ERROR [connector|task-0] Producer failure (io.debezium.pipeline.ErrorHandler:35)
      [Worker-0e19902a182ab6160] java.lang.UnsupportedOperationException: Read only connection requires GTID_MODE to be ON
      [Worker-0e19902a182ab6160] 	at io.debezium.connector.mysql.MySqlChangeEventSourceFactory.getIncrementalSnapshotChangeEventSource(MySqlChangeEventSourceFactory.java:100)
      [Worker-0e19902a182ab6160] 	at io.debezium.connector.mysql.MySqlChangeEventSourceFactory.getIncrementalSnapshotChangeEventSource(MySqlChangeEventSourceFactory.java:29)
      [Worker-0e19902a182ab6160] 	at io.debezium.pipeline.ChangeEventSourceCoordinator.initStreamEvents(ChangeEventSourceCoordinator.java:185)
      [Worker-0e19902a182ab6160] 	at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:172)
      [Worker-0e19902a182ab6160] 	at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:141)
      [Worker-0e19902a182ab6160] 	at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
      [Worker-0e19902a182ab6160] 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      [Worker-0e19902a182ab6160] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      [Worker-0e19902a182ab6160] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      [Worker-0e19902a182ab6160] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      [Worker-0e19902a182ab6160] 	at java.base/java.lang.Thread.run(Thread.java:829) 

       

              Unassigned Unassigned
              averbaq Alejandro Vera-Baquero (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: