Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6636

Include only certain columns in JDBC sink connector

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Done
    • Icon: Major Major
    • 2.5.0.Beta1
    • None
    • jdbc-connector
    • None
    • False
    • None
    • False
    • 0
    • 0% 0%

      Feature request or enhancement

      Which use case/requirement will be addressed by the proposed feature?

      Allow the JDBC sink connector to specify whitelisted columns, to allow only a subset of columns to be inserted into the table.

      Say I have 2 pre-defined Postgres tables. The source and sink table have the same schema, which works perfectly when using the Postgres source connector and JDBC sink connector. Now let's say that the source table adds a column, which I don't require in my sink table. I want to keep schema.evolution set to none, as I don't want my sink table to divert from it's original schema.

      Now the JDBC sink connector fails to insert any data, with the following error:

      Caused by: java.sql.SQLException: Cannot alter table db.public.sink because schema evolution is disabled 

       

      Is it possible to select which columns I want to extract data from within my JDBC config. Or would I have to instead add my own custom SMT to get around this for the timebeing?

      Implementation ideas (optional)

      Allow the following configuration options in the JDBC sink connector, that are available in the Postgres source connector:

      • column.exclude.list
      • column.include.list

            anmohant Anisha Mohanty
            camward Cameron Ward (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: