Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6606

Garbled characters appear when the field value is Chinese characters

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Won't Do
    • Icon: Major Major
    • None
    • 2.3.0.Final
    • None
    • False
    • None
    • False
    • Important

      Bug report

      What Debezium connector do you use and what version?

      Docker image  quay.io/debezium/connect:2.3

      What is the connector configuration?

      {
          "connector.class": "io.debezium.connector.mysql.MySqlConnector",
          "database.user": "root",
          "database.server.id": "1000",
          "schema.history.internal.kafka.bootstrap.servers": "192.168.22.22:9092",
          "database.port": "3306",
          "include.schema.changes": "true",
          "topic.prefix": "1000",
          "schema.history.internal.kafka.topic": "1000.test.history",
          "include.schema.comments": "false",
          "database.hostname": "192.168.22.22",
          "database.password": "********",
          "name": "1000.test",
          "skip.messages.without.change": "true",
          "table.include.list": "test.t_auth",
          "include.query": "true",
          "database.include.list": "test"
      } 

      What is the captured database version and mode of depoyment?

       

      Both versions have this problem:

      quay.io/debezium/example-mysql:2.3

      Mysql:8.0

       

      What behaviour do you expect?

      Chinese characters are synchronized normally.

      What behaviour do you see?

      Create Table SQL:

       

      CREATE TABLE `t_auth` (
          `Id` INT NOT NULL AUTO_INCREMENT,
          `SystemId` INT NOT NULL,
          `AuthClassifyId` INT NOT NULL,
          `Name` VARCHAR ( 20 ) CHARACTER 
          SET utf8mb4 COLLATE utf8mb4_0900_ai_ci NOT NULL,
          `Code` VARCHAR ( 20 ) CHARACTER 
          SET utf8mb4 COLLATE utf8mb4_0900_ai_ci DEFAULT NULL,
          `Order` INT DEFAULT NULL,
          `IsBtn` TINYINT ( 1 ) DEFAULT '0',
          `IsEnable` bit ( 1 ) NOT NULL,
          `CreateTime` datetime NOT NULL,
          `CreateUserId` INT NOT NULL,
          `UpdateUserId` INT DEFAULT NULL,
          `UpdateTime` datetime DEFAULT NULL,
          `IsDeleted` bit ( 1 ) NOT NULL DEFAULT b'0',
          `Comment` VARCHAR ( 256 ) CHARACTER 
          SET utf8mb4 COLLATE utf8mb4_0900_ai_ci DEFAULT NULL,
          PRIMARY KEY ( `Id` ) USING BTREE 
          ) ENGINE = INNODB AUTO_INCREMENT = 87 DEFAULT CHARSET = utf8mb4 COLLATE = utf8mb4_0900_ai_ci ROW_FORMAT = DYNAMIC

       

       

      Insert data:

       

           INSERT INTO `test`.`t_auth` (
          `Id`,
          `SystemId`,
          `AuthClassifyId`,
          `Name`,
          `Code`,
          `Order`,
          `IsBtn`,
          `IsEnable`,
          `CreateTime`,
          `CreateUserId`,
          `UpdateUserId`,
          `UpdateTime`,
          `IsDeleted`,
          `Comment` 
      )
      VALUES
          (
              86,
              1,
              57,
              '系统配置权限',
              NULL,
              0,
              0,
              b'1',
              '2022-11-04 11:39:03',
              0,
              0,
              '2022-11-11 09:25:08',
          b'0',
          NULL) 

       

       

       

      Debezium push to kafka message,query field:

      "payload": {
            "before": null,
            "after": {
               "Id": 86,
               "SystemId": 1,
               "AuthClassifyId": 57,
               "Name": "系统配置权限",
               "Code": null,
               "Order": 0,
               "IsBtn": 0,
               "IsEnable": true,
               "CreateTime": 1667561943000,
               "CreateUserId": 0,
               "UpdateUserId": 0,
               "UpdateTime": 1668158708000,
               "IsDeleted": false,
               "Comment": null
            },
            "source": {
               "version": "2.3.0.Alpha1",
               "connector": "mysql",
               "name": "1000",
               "ts_ms": 1687741437000,
               "snapshot": "false",
               "db": "test",
               "sequence": null,
               "table": "t_auth",
               "server_id": 223344,
               "gtid": null,
               "file": "mysql-bin.000008",
               "pos": 16904009,
               "row": 0,
               "thread": 945,
               "query": "INSERT INTO `test`.`t_auth` (`Id`, `SystemId`, `AuthClassifyId`, `Name`, `Code`, `Order`, `IsBtn`, `IsEnable`, `CreateTime`, `CreateUserId`, `UpdateUserId`, `UpdateTime`, `IsDeleted`, `Comment`) VALUES (86, 1, 57, '������������������', NULL, 0, 0, b'1', '2022-11-04 11:39:03', 0, 0, '2022-11-11 09:25:08', b'0', NULL)"
            },
            "op": "c",
            "ts_ms": 1687741437005,
            "transaction": null
         } 

        1. image-2023-06-26-08-54-52-035.png
          22 kB
          dean yan
        2. image-2023-06-26-09-03-18-952.png
          18 kB
          dean yan
        3. image-2023-06-26-09-04-21-080.png
          4 kB
          dean yan
        4. image-2023-07-06-11-35-26-844.png
          478 kB
          dean yan
        5. image-2023-07-06-11-35-34-176.png
          0.3 kB
          dean yan

              ywyuewei Harvey Yue (Inactive)
              whuanle dean yan (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: