Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-9123

Support for partial_json mode and message in MySQL connector

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Unresolved
    • Icon: Major Major
    • Backlog
    • None
    • mysql-connector
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      3.1.1

      What is the connector configuration?

      Source connector

      {
          "name": "employee-connector",
          "config":

      {         "connector.class": "io.debezium.connector.mysql.MySqlConnector",         "database.hostname": "10.0.0.1",         "database.port": "3306",         "database.user": "admin",         "database.password": "secret",         "database.ssl.mode": "preferred",         "database.server.id": "1234",         "topic.prefix": "dbserver2",         "database.include.list": "testcdc",         "table.include.list": "testcdc.employee",         "snapshot.mode": "always",         "snapshot.locking.mode": "extended",         "schema.history.internal.kafka.bootstrap.servers": "kafka:9092",         "schema.history.internal.kafka.topic": "schema-changes.testcdc",     "log.level": "DEBUG"     }

      }

      What is the captured database version and mode of deployment?

      (E.g. on-premises, with a specific cloud provider, etc.)

      I am using container on OCI VM using HeatWave MySQL on OCI as both source and target database. I have tested on MySQL 8.X and 9.X

      What behavior do you expect?

      Debezium CDC Connector for MySQL uses MySQL Replication client API, I expect MySQL user account would only need to have SELECT, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT (without RELOAD) privileges in order to read from MySQL binlog. 

      What behavior do you see?

      UPDATE events are not extracted by Debezium CDC connector for MySQL using MySQL user account without RELOAD privilege, INSERT/DELETE events are fine though. 

       

      Do you see the same behaviour using the latest released Debezium version?

      (Ideally, also verify with latest Alpha/Beta/CR version)

      I am using the latest version of 3.1 Debezium connector

      Do you have the connector logs, ideally from start till finish?

      (You might be asked later to provide DEBUG/TRACE level log)

      Yes

      How to reproduce the issue using our tutorial deployment?

      <Your answer>

      You would only need to have a standalone MySQL server (as source and target) with an user account without the RELOAD privilege to reproduce the behavior. 

      Feature request or enhancement

      For feature requests or enhancements, provide this information, please:

      Which use case/requirement will be addressed by the proposed feature?

      • Data distribution from OCI HeatWave MySQL to other sources
      • Enforce tighter security model without RELOAD privileges to user account unnecessarily

      I am interested to understand why RELOAD is required by Debezium CDC connector to extract binlog events, specifically what is the exact command required? (FLUSH BINARY LOGS?)

      Implementation ideas (optional)

      <Your answer>

       

        1. binary-log.005358
          5 kB
          Ryan Kuan
        2. connect.log
          470 kB
          Ryan Kuan
        3. OCI-HeatWave-MySQL-source.png
          174 kB
          Ryan Kuan
        4. Screenshot 2025-06-09 at 7.40.36 PM.png
          72 kB
          Ryan Kuan
        5. Screenshot 2025-06-09 at 7.44.37 PM.png
          66 kB
          Ryan Kuan
        6. Screenshot 2025-06-13 at 11.09.14 AM.png
          19 kB
          Ryan Kuan
        7. self-hosted-MySQL-source.png
          163 kB
          Ryan Kuan
        8. topic.log
          15 kB
          Ryan Kuan

              Unassigned Unassigned
              kuanrcl Ryan Kuan (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: