Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7587

How to specify the "capture_instance" parameter for Debezuim

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Unresolved
    • Icon: Major Major
    • 2.7.0.Alpha2
    • None
    • sqlserver-connector
    • None
    • False
    • None
    • False
    • 0
    • 0% 0%

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

       

      Debezium synchronizes data with SQL Server DB by enabling the CDC function in the tables of SQL Server. Multiple "capture_instance" can be created, and each capture_instance can specify different fields. The debezium connector seems to have no parameters to specify capture_instance.

      For example, the "dbo. TEST 01" table in the SQL server database has three fields (id, name, age). To enable the CDC function for this table, create a "capture_instance" named custom1 and capture the fields (id, name, age). Debezium will store the structure of this table in the "history topic" of Kafka. Create another "capture_instance" named custom2, capture the field as (id), and debezium will update the table structure of this table in Kafka. If debezium has specified the "capture_instance" parameter configuration, can the table structure not be updated.

      This is the result I want

       

       

      Configuration parameters for SQL Server DB:

      https://learn.microsoft.com/en-us/sql/relational-databases/system-stored-procedures/sys-sp-cdc-enable-table-transact-sql?view=sql-server-ver16#----capture_instance 

      Debezium's explanation for table updates : 

      https://debezium.io/documentation/reference/2.5/connectors/sqlserver.html#sqlserver-schema-evolution

       

      What Debezium connector do you use and what version?

      I am using Debezium Sqlserver Source connector, and the version is 2.3.0.

      What is the connector configuration?

      {
          "name": "sqlserver-connector-94",
          "config":

      {         "connector.class": "io.debezium.connector.sqlserver.SqlServerConnector",         "tasks.max": 1 ,         "database.encrypt": "false",         "database.trustServerCertificate": "false",         "database.port": "1433",         "database.user": "sa",         "database.password": "PASSWORD",         "database.hostname": "192.168.1.97",         "database.names": "test",         "schema.include.list": "dbo",         "table.include.list": "dbo.TEST01",         "transforms.Reroute.type": "io.debezium.transforms.ByLogicalTableRouter",         "schema.history.internal.kafka.bootstrap.servers": "192.168.1.13:9092,192.168.1.13:9093,192.168.1.13:9094",         "schema.history.internal.kafka.topic": "JHDL-TOPIC-CDC4S-LOG-dbhistory-94",         "transforms": "Reroute",         "database.server.name": "JHDL-TOPIC-CDC4S-LOG",         "topic.prefix": "JHDL-TOPIC-CDC4S-LOG",         "transforms.Reroute.topic.regex": "JHDL-TOPIC-CDC4S-LOG.(.*)",         "transforms.Reroute.topic.replacement": "JHDL-TOPIC-CDC4S-LOG-94",         "snapshot.mode": "schema_only",         "decimal.handling.mode": "string",         "snapshot.isolation.mode": "snapshot",         "database.applicationIntent":"ReadOnly"     }

      }

      What is the captured database version and mode of depoyment?

       

      SQLServer
      Microsoft SQL Server 2022 (RTM) - 16.0.1000.6 (X64)   Oct  8 2022 05:58:25   Copyright (C) 2022 Microsoft Corporation  Developer Edition (64-bit) on Windows Server 2016 Datacenter 10.0 <X64> (Build 14393: ) (Hypervisor) 

       

      KAFKA Version:  2.12-2.4.1

      What behaviour do you expect?

      I expect Debezium to have the ability to specify the "capture_instance" parameter configuration, so that when I create multiple "capture_instances" in SQL Server, Debezium will not update the table structure

      What behaviour do you see?

      <Your answer>

      Do you see the same behaviour using the latest relesead Debezium version?

      (Ideally, also verify with latest Alpha/Beta/CR version)

       

      Haven't seen

      Do you have the connector logs, ideally from start till finish?

      (You might be asked later to provide DEBUG/TRACE level log)

      This is not an error .

      How to reproduce the issue using our tutorial deployment?

      <Your answer>

      Feature request or enhancement

      For feature requests or enhancements, provide this information, please:

      Debezium synchronizes data with SQL Server DB by enabling the CDC function in the tables of SQL Server. Multiple "capture_instance" can be created, and each capture_instance can specify different fields. The debezium connector seems to have no parameters to specify capture_instance.

      For example, the "dbo. TEST 01" table in the SQL server database has three fields (id, name, age). To enable the CDC function for this table, create a "capture_instance" named custom1 and capture the fields (id, name, age). Debezium will store the structure of this table in the "history topic" of Kafka. Create another "capture_instance" named custom2, capture the field as (id), and debezium will update the table structure of this table in Kafka. If debezium has specified the "capture_instance" parameter configuration, can the table structure not be updated.

      This is the result I want

       

      Which use case/requirement will be addressed by the proposed feature?

      <Your answer>

      Implementation ideas (optional)

       

      Not currently available

       

       
       
       
       

            Unassigned Unassigned
            180-004-251 德胜 陈
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: