Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6720

Toasted UUID array is not properly processed

XMLWordPrintable

    • False
    • None
    • False
    • Important

      The connector is failing with the following error when unchanged column of type UUID array is toasted:

      Caused by: org.apache.kafka.connect.errors.DataException: Invalid value: null used for required field: "tags", schema type: ARRAY
          at org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:220)
          at org.apache.kafka.connect.data.Struct.validate(Struct.java:233)
          at org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:250)
          at org.apache.kafka.connect.data.Struct.put(Struct.java:216)
          at org.apache.kafka.connect.data.Struct.put(Struct.java:203)
          at io.debezium.data.Envelope.update(Envelope.java:323)
          at io.debezium.relational.RelationalChangeRecordEmitter.emitUpdateRecord(RelationalChangeRecordEmitter.java:112)
          at io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:53)
          at io.debezium.connector.postgresql.PostgresChangeRecordEmitter.emitChangeRecords(PostgresChangeRecordEmitter.java:96)
          at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:209)
          ... 16 more 

      https://issues.redhat.com/browse/DBZ-5936 fixed the similar issue for different array types like int, bigint, json etc. However, the issue still persists for column of type UUID array.

      The UUID type is internally mapped to string type in Kafka Connect.

      // e.g. tags is a column with type uuid[] and is represented as follows in the kafka record
      
      "tags": [
          {
            "string": "88fb0840-9370-4e15-8d83-05cbe74201d8"
          },
          {
            "string": "89883a5e-902e-439f-b966-dc096afe6ff9"
          }
      ] 
      
      // corresponding schema 
      {
            "name": "tags",
            "type": {
              "items": [
                "null",
                {
                  "connect.name": "io.debezium.data.Uuid",
                  "connect.version": 1,
                  "type": "string"
                }
              ],
              "type": "array"
            }
      }

       

       

            Unassigned Unassigned
            rdangwal Rajendra Dangwal
            Rajendra Dangwal
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: