Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3974

Misc. documentation changes for the Debezium MySQL connector

    XMLWordPrintable

Details

    Description

      I've noticed a few things that should be updated in the MySQL connector docs:

      • Schema change events contain both the raw DDL as well as a structured representation of the table schema at that point in time (since DBZ-1865, added in 1.5); here's an example (note the new tableChanges field; this field is included in the examples currently used in the Db2, Oracle, and SQL Server connector docs):
      {
          "schema": {
              "type": "struct",
              "fields": [
                  {
                      "type": "string",
                      "optional": false,
                      "field": "databaseName"
                  }
              ],
              "optional": false,
              "name": "io.debezium.connector.mysql.SchemaChangeKey"
          },
          "payload": {
              "databaseName": "inventory"
          }
      }{
          "schema": {
              "type": "struct",
              "fields": [
                  {
                      "type": "struct",
                      "fields": [
                          {
                              "type": "string",
                              "optional": false,
                              "field": "version"
                          },
                          {
                              "type": "string",
                              "optional": false,
                              "field": "connector"
                          },
                          {
                              "type": "string",
                              "optional": false,
                              "field": "name"
                          },
                          {
                              "type": "int64",
                              "optional": false,
                              "field": "ts_ms"
                          },
                          {
                              "type": "string",
                              "optional": true,
                              "name": "io.debezium.data.Enum",
                              "version": 1,
                              "parameters": {
                                  "allowed": "true,last,false"
                              },
                              "default": "false",
                              "field": "snapshot"
                          },
                          {
                              "type": "string",
                              "optional": false,
                              "field": "db"
                          },
                          {
                              "type": "string",
                              "optional": true,
                              "field": "sequence"
                          },
                          {
                              "type": "string",
                              "optional": true,
                              "field": "table"
                          },
                          {
                              "type": "int64",
                              "optional": false,
                              "field": "server_id"
                          },
                          {
                              "type": "string",
                              "optional": true,
                              "field": "gtid"
                          },
                          {
                              "type": "string",
                              "optional": false,
                              "field": "file"
                          },
                          {
                              "type": "int64",
                              "optional": false,
                              "field": "pos"
                          },
                          {
                              "type": "int32",
                              "optional": false,
                              "field": "row"
                          },
                          {
                              "type": "int64",
                              "optional": true,
                              "field": "thread"
                          },
                          {
                              "type": "string",
                              "optional": true,
                              "field": "query"
                          }
                      ],
                      "optional": false,
                      "name": "io.debezium.connector.mysql.Source",
                      "field": "source"
                  },
                  {
                      "type": "string",
                      "optional": true,
                      "field": "databaseName"
                  },
                  {
                      "type": "string",
                      "optional": true,
                      "field": "schemaName"
                  },
                  {
                      "type": "string",
                      "optional": true,
                      "field": "ddl"
                  },
                  {
                      "type": "array",
                      "items": {
                          "type": "struct",
                          "fields": [
                              {
                                  "type": "string",
                                  "optional": false,
                                  "field": "type"
                              },
                              {
                                  "type": "string",
                                  "optional": false,
                                  "field": "id"
                              },
                              {
                                  "type": "struct",
                                  "fields": [
                                      {
                                          "type": "string",
                                          "optional": true,
                                          "field": "defaultCharsetName"
                                      },
                                      {
                                          "type": "array",
                                          "items": {
                                              "type": "string",
                                              "optional": false
                                          },
                                          "optional": true,
                                          "field": "primaryKeyColumnNames"
                                      },
                                      {
                                          "type": "array",
                                          "items": {
                                              "type": "struct",
                                              "fields": [
                                                  {
                                                      "type": "string",
                                                      "optional": false,
                                                      "field": "name"
                                                  },
                                                  {
                                                      "type": "int32",
                                                      "optional": false,
                                                      "field": "jdbcType"
                                                  },
                                                  {
                                                      "type": "int32",
                                                      "optional": true,
                                                      "field": "nativeType"
                                                  },
                                                  {
                                                      "type": "string",
                                                      "optional": false,
                                                      "field": "typeName"
                                                  },
                                                  {
                                                      "type": "string",
                                                      "optional": true,
                                                      "field": "typeExpression"
                                                  },
                                                  {
                                                      "type": "string",
                                                      "optional": true,
                                                      "field": "charsetName"
                                                  },
                                                  {
                                                      "type": "int32",
                                                      "optional": true,
                                                      "field": "length"
                                                  },
                                                  {
                                                      "type": "int32",
                                                      "optional": true,
                                                      "field": "scale"
                                                  },
                                                  {
                                                      "type": "int32",
                                                      "optional": false,
                                                      "field": "position"
                                                  },
                                                  {
                                                      "type": "boolean",
                                                      "optional": true,
                                                      "field": "optional"
                                                  },
                                                  {
                                                      "type": "boolean",
                                                      "optional": true,
                                                      "field": "autoIncremented"
                                                  },
                                                  {
                                                      "type": "boolean",
                                                      "optional": true,
                                                      "field": "generated"
                                                  }
                                              ],
                                              "optional": false,
                                              "name": "io.debezium.connector.schema.Column"
                                          },
                                          "optional": false,
                                          "field": "columns"
                                      }
                                  ],
                                  "optional": false,
                                  "name": "io.debezium.connector.schema.Table",
                                  "field": "table"
                              }
                          ],
                          "optional": false,
                          "name": "io.debezium.connector.schema.Change"
                      },
                      "optional": false,
                      "field": "tableChanges"
                  }
              ],
              "optional": false,
              "name": "io.debezium.connector.mysql.SchemaChangeValue"
          },
          "payload": {
              "source": {
                  "version": "1.7.0.Alpha1",
                  "connector": "mysql",
                  "name": "dbserver1",
                  "ts_ms": 1631007377173,
                  "snapshot": "false",
                  "db": "inventory",
                  "sequence": null,
                  "table": "customers",
                  "server_id": 223344,
                  "gtid": null,
                  "file": "mysql-bin.000003",
                  "pos": 219,
                  "row": 0,
                  "thread": null,
                  "query": null
              },
              "databaseName": "inventory",
              "schemaName": null,
              "ddl": "alter table customers add column middle_name VARCHAR(2000)",
              "tableChanges": [
                  {
                      "type": "ALTER",
                      "id": "\"inventory\".\"customers\"",
                      "table": {
                          "defaultCharsetName": "latin1",
                          "primaryKeyColumnNames": [
                              "id"
                          ],
                          "columns": [
                              {
                                  "name": "id",
                                  "jdbcType": 4,
                                  "nativeType": null,
                                  "typeName": "INT",
                                  "typeExpression": "INT",
                                  "charsetName": null,
                                  "length": 11,
                                  "scale": null,
                                  "position": 1,
                                  "optional": false,
                                  "autoIncremented": true,
                                  "generated": true
                              },
                              {
                                  "name": "first_name",
                                  "jdbcType": 12,
                                  "nativeType": null,
                                  "typeName": "VARCHAR",
                                  "typeExpression": "VARCHAR",
                                  "charsetName": "latin1",
                                  "length": 255,
                                  "scale": null,
                                  "position": 2,
                                  "optional": false,
                                  "autoIncremented": false,
                                  "generated": false
                              },
                              {
                                  "name": "last_name",
                                  "jdbcType": 12,
                                  "nativeType": null,
                                  "typeName": "VARCHAR",
                                  "typeExpression": "VARCHAR",
                                  "charsetName": "latin1",
                                  "length": 255,
                                  "scale": null,
                                  "position": 3,
                                  "optional": false,
                                  "autoIncremented": false,
                                  "generated": false
                              },
                              {
                                  "name": "email",
                                  "jdbcType": 12,
                                  "nativeType": null,
                                  "typeName": "VARCHAR",
                                  "typeExpression": "VARCHAR",
                                  "charsetName": "latin1",
                                  "length": 255,
                                  "scale": null,
                                  "position": 4,
                                  "optional": false,
                                  "autoIncremented": false,
                                  "generated": false
                              },
                              {
                                  "name": "middle_name",
                                  "jdbcType": 12,
                                  "nativeType": null,
                                  "typeName": "VARCHAR",
                                  "typeExpression": "VARCHAR",
                                  "charsetName": "latin1",
                                  "length": 2000,
                                  "scale": null,
                                  "position": 5,
                                  "optional": true,
                                  "autoIncremented": false,
                                  "generated": false
                              }
                          ]
                      }
                  }
              ]
          }
      }
      
      • The reference to the database.server.name option in the same section should be an actual link to the property in the docs (Db2, MySQL, Oracle, SQL Server).
      • The Topic names section only discusses names of change event topics; for sake of completeness, the default topic naming should also refer to naming for:
        • Schema change topics (Db2, MySQL, Oracle, SQL Server).
        • Internal DB history topics (Db2, MySQL, Oracle, SQL Server).
        • Transaction metadata topics (Db2, MySQL, PG, Oracle, SQL Server)

      Attachments

        Activity

          People

            broldan@redhat.com Robert Roldan
            gmorling@redhat.com Gunnar Morling
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: