Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1051

Inconsistent schema name in streaming and snapshotting phase

    XMLWordPrintable

Details

    Description

      I have defined in connector's config "database.dbname": "inventory" while in fact the database name is INVENTORY. It worked since SQL Server seems to be case insensitive.

      All records created during snapshot phase have schema name mssql_server.INVENTORY.dbo.products.Key or mssql_server.INVENTORY.dbo.products.Value.
      While all records created during streaming have schema name mssql_server.inventory.dbo.products.Key or mssql_server.inventory.dbo.products.Value.

      An event from snapshot phase (key and value):

      {
          "schema": {
              "type": "struct",
              "fields": [
                  {
                      "type": "int32",
                      "optional": false,
                      "field": "id"
                  }
              ],
              "optional": false,
              "name": "mssql_server.INVENTORY.dbo.products.Key"
          },
          "payload": {
              "id": 109
          }
      }
      
      {
          "schema": {
              "type": "struct",
              "fields": [
                  {
                      "type": "int32",
                      "optional": false,
                      "field": "id"
                  },
                  {
                      "type": "string",
                      "optional": false,
                      "field": "name"
                  },
                  {
                      "type": "string",
                      "optional": true,
                      "field": "description"
                  },
                  {
                      "type": "double",
                      "optional": true,
                      "field": "weight"
                  }
              ],
              "optional": true,
              "name": "mssql_server.INVENTORY.dbo.products.Value"
          },
          "payload": {
              "id": 109,
              "name": "spare tire",
              "description": "24 inch spare tire",
              "weight": 22.2
          }
      }
      

      An event from streaming phase (key and value):

      {
          "schema": {
              "type": "struct",
              "fields": [
                  {
                      "type": "int32",
                      "optional": false,
                      "field": "id"
                  }
              ],
              "optional": false,
              "name": "mssql_server.inventory.dbo.products.Key"
          },
          "payload": {
              "id": 110
          }
      }
      {
          "schema": {
              "type": "struct",
              "fields": [
                  {
                      "type": "int32",
                      "optional": false,
                      "field": "id"
                  },
                  {
                      "type": "string",
                      "optional": false,
                      "field": "name"
                  },
                  {
                      "type": "string",
                      "optional": true,
                      "field": "description"
                  },
                  {
                      "type": "double",
                      "optional": true,
                      "field": "weight"
                  }
              ],
              "optional": true,
              "name": "mssql_server.inventory.dbo.products.Value"
          },
          "payload": {
              "id": 110,
              "name": "book",
              "description": "a book",
              "weight": 0.6
          }
      }
      

      If Avro format is used instead of JSON, the connector fails since it tries to register an incompatible schema in Schema Registry (the schema itself is the same but the schema namespace is different - schema registry is case sensitive).

      Attachments

        Activity

          People

            jpechane Jiri Pechanec
            grzegorz.kolakowski Grzegorz KoĊ‚akowski (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: