Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-8224

JDBC sink doesn't include fields as per documentation

XMLWordPrintable

      Bug report

      What Debezium connector do you use and what version?

      debezium-connector-jdbc

      What is the connector configuration?

       

      {
          "name": "subaccount-sink",
          "config": {
              "topics": "subaccount",
              "table.name.format": "test_subaccount",
              "connector.class": "io.debezium.connector.jdbc.JdbcSinkConnector",
              "tasks.max": "1",
              "connection.url": "jdbc:postgresql://postgres:5432/postgres",
              "connection.username": "postgres",
              "connection.password": "xxxx",
              "insert.mode": "upsert",
              "delete.enabled": "true",
              "primary.key.mode": "record_key",
              "database.time_zone": "UTC",
              "connection.pool.min_size" : 20,
              "connection.pool.max_size" : 20,
              "batch.size": "50",
              "schema.evolution":"basic",
              "quote.identifiers": "true",
              "field.include.list": "SUBACCOUNT_ID,NAME,INTRODUCED_ON",
              "primary.key.fields": "SUBACCOUNT_ID",    }
      }
       

      What is the captured database version and mode of deployment?

      postgres 15

      What behavior do you expect?

      the fields listed to be captured

      according to the documentation: "Fully-qualified names for fields are of the form fieldName or topicName:fieldName."

      https://debezium.io/documentation/reference/stable/connectors/jdbc.html#jdbc-property-field-include-list

      but the code: https://github.com/debezium/debezium-connector-jdbc/blob/32cd663ca38d0533e919dc087a41b24daa82b17b/src/main/java/io/debezium/connector/jdbc/filter/FieldFilterFactory.java#L32

      only checks for two part names and checks both parts

      What behavior do you see?

      rows captured and an exception saying that there were no key columns

      Do you see the same behaviour using the latest released Debezium version?

      yes

      Do you have the connector logs, ideally from start till finish?

      N/A

      How to reproduce the issue using our tutorial deployment?

      https://debezium.io/documentation/reference/stable/connectors/jdbc.html

      just add the  "field.include.list": 

      Feature request or enhancement

      enhancement?

      Which use case/requirement will be addressed by the proposed feature?

      filtering what fields/columns are included in the database

      Implementation ideas (optional)

      https://github.com/debezium/debezium-connector-jdbc/pull/90

      https://github.com/debezium/debezium/pull/5845

       

              Unassigned Unassigned
              msillencefnz Martin Sillence
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: