Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-175

table.whitelist option update for an existing connector doesn't work

    XMLWordPrintable

Details

    • Feature Request
    • Resolution: Done
    • Major
    • 0.9.0.CR1
    • 0.3.6
    • mysql-connector
    • None
    • 0
    • 0% 0%
    • Hide
      A first implementation of whitelist changes has been merged and will be part of Debezium 0.9.0.CR1. Please see the comments above for some things to be aware of when using this: there shouldn't be done any schema changes while the whitelist/blacklist changes are processed and the connector should only be shut down *after* that processing has happened and a single binlog reader is running. There is DBZ-1091 for addressing these issues. Support for whitelist/blacklist changes is disabled by default for the time being and should be considered an experimental feature until the implementation has stabilized.
      Show
      A first implementation of whitelist changes has been merged and will be part of Debezium 0.9.0.CR1. Please see the comments above for some things to be aware of when using this: there shouldn't be done any schema changes while the whitelist/blacklist changes are processed and the connector should only be shut down *after* that processing has happened and a single binlog reader is running. There is DBZ-1091 for addressing these issues. Support for whitelist/blacklist changes is disabled by default for the time being and should be considered an experimental feature until the implementation has stabilized.

    Description

      We found impossible to alter config of an existed connector.

      Requirement :
      – ability to enhance table.whitelist config property of an existed container with time.

      Steps we did :

      1. create new connector with only one table presented in table.whitelist property
      2. verify initial snapshot has been taken
      3. verify new changes were read properly from binlog
      4. PUT new config using PUT /connectors/(string: name)/config standard kafka-connect method. Config was exactly the same as we used at first step with just new table addition to table.whitelist property
      5. we noticed that config update caused graceful restart/rebalance of all existed tasks. So, the expectation was that changes were applied immediately after all tasks started again.

      Existed Behaviour :
      – nothing changed : topic wasn't created for newly added table and initial snapshot haven't been made for it

      Expected Behavior :
      – new addition to table.whitelist property had to cause initial snapshot read for new tables with appropriate topic creation and data population from binlog

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem that impairs or prevents the functions of the product. DBZ-552 history records were filtered by white/black list before storing to kafka db history topic

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is related to

          Feature Request - Feature requests from customers and/or users DBZ-165 Snapshot new tables after initial startup

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Task - Represents a small unit of work that is not end-user facing. DBZ-1091 Follow-up tasks for filter config change handling

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Task - Represents a small unit of work that is not end-user facing. DBZ-1977 Document snapshot.new.tables configuration option

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) DBZ-66 Support ad hoc snapshots on MySQL connector

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              moirat Moira Tagle (Inactive)
              anton_nazaruk Anton Nazaruk (Inactive)
              Votes:
              8 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: