Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4941

Handle toasted String array

    XMLWordPrintable

Details

    • False
    • None
    • False

    Description

      hi theres 

      When importing data from postgresql, sometimes the following specific error occurs.

       

      Caused by: org.apache.kafka.connect.errors.DataException: Invalid value: null used for required field: "external_invoice_urls", schema type: ARRAY
	at org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:220)
	at org.apache.kafka.connect.data.Struct.validate(Struct.java:233)
	at org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:253)
	at org.apache.kafka.connect.data.Struct.put(Struct.java:216)
	at org.apache.kafka.connect.data.Struct.put(Struct.java:203)
	at io.debezium.data.Envelope.update(Envelope.java:323)
	at io.debezium.relational.RelationalChangeRecordEmitter.emitUpdateRecord(RelationalChangeRecordEmitter.java:111)
	at io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:52)
	at io.debezium.connector.postgresql.PostgresChangeRecordEmitter.emitChangeRecords(PostgresChangeRecordEmitter.java:97)
	at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:215)
	... 17 more

       

      Since the external_invoice_urls column is a non-nullable value, there is no default value, and shouldn't a value always be passed in?

       

      heres is table schema 

      SELECT column_name, is_nullable, data_type
FROM information_schema.COLUMNS
WHERE TABLE_NAME = 'reservations';

       

       

      Attachments

        Issue Links

          relates to

          Bug - A problem that impairs or prevents the functions of the product. DBZ-5935 Toasted binary array is not correctly processed

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. DBZ-5936 Toasted json/int/bigint arrays are not properly processed

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. DBZ-5944 Toasted varchar array is not correctly processed

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              Unassigned Unassigned
              jaegwon.seo wornjs seo (Inactive)
              Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: