Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-9218

Support user-defined types

XMLWordPrintable

    • 8
    • False
    • Hide

      None

      Show
      None
    • False

      Like other relational databases, Oracle supports user-defined types so that you can define a column that consists of things like XML or nested objects. For example:

      CREATE OR REPLACE TYPE address_type 
        AS OBJECT (street VARCHAR2(100), city varchar2(50), postal_code varchar2(50));
      
      CREATE TABLE employees (
        id number(9,0) primary key, 
        name varchar2(50), 
        address varchar2(50))
      

      During the table metadata fetch, Oracle only provides us DDL as follows:

        CREATE TABLE "DEBEZIUM"."EMPLOYEES"
         (    "ID" NUMBER(9,0),
          "NAME" VARCHAR2(50),
          "ADDRESS" "DEBEZIUM"."ADDRESS_TYPE" ,
           PRIMARY KEY ("ID")
        USING INDEX  ENABLE,
           SUPPLEMENTAL LOG DATA (ALL) COLUMNS
         ) ;, tables=[columns: {
        ID NUMBER(9, 0) NOT NULL
        NAME VARCHAR2(50) DEFAULT VALUE NULL
        ADDRESS ADDRESS_TYPE(1) DEFAULT VALUE NULL
      }
      

      And later during the snapshot processing phase of this DDL, it reports:

      2025-07-11 10:49:17,631 DEBUG  Oracle||snapshot  Building schema for column ADDRESS of type 1111 named ADDRESS_TYPE with constraints (1,Optional.empty)   [io.debezium.connector.oracle.OracleValueConverters]
      2025-07-11 10:49:17,631 DEBUG  Oracle||snapshot  JdbcValueConverters returned 'null' for column 'ADDRESS'   [io.debezium.connector.oracle.OracleValueConverters]
      2025-07-11 10:49:17,631 WARN   Oracle||snapshot  Unexpected JDBC type '1111' for column 'ADDRESS' that will be ignored   [io.debezium.relational.TableSchemaBuilder]
      

      This is all expected because unknown data types are not things that the connector serializes, and instead, user data types are simply ignored.

      This leads to a couple points:

      How do we intend to represent UDTs in the relational model?
      During streaming by default, UDTs are provided as XML operations, but we need to check if the serialization is database parameter-driven. Therefore if we opt to represent user-defined types as anything other than XML/VARCHAR2 data types, then we would likely need to also think about introducing some conversion layer at streaming.

      Oracle XML parser configuration
      This also raises questions about Oracle XML support. The Oracle XML data types require setting the following explicitly on the JVM process:

      -Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl
      

      This isn't something that can likely be done easily in cloud environments, and it's one reason why XML support has remained incubating. The Oracle XML libraries dictate this requirement.

      Ideally I think it would worthwhile to see if we can avoid the `-D` argument or be able to dictate this via connector configuration, and if so, that makes that point moot and also stabilizes the XML support, while giving us a path forward to consider UDTs emitted as XML.

              Unassigned Unassigned
              ccranfor@redhat.com Chris Cranford
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: