Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-175

table.whitelist option update for an existing connector doesn't work

    XMLWordPrintable

Details

    • Hide
      A first implementation of whitelist changes has been merged and will be part of Debezium 0.9.0.CR1. Please see the comments above for some things to be aware of when using this: there shouldn't be done any schema changes while the whitelist/blacklist changes are processed and the connector should only be shut down *after* that processing has happened and a single binlog reader is running. There is DBZ-1091 for addressing these issues. Support for whitelist/blacklist changes is disabled by default for the time being and should be considered an experimental feature until the implementation has stabilized.
      Show
      A first implementation of whitelist changes has been merged and will be part of Debezium 0.9.0.CR1. Please see the comments above for some things to be aware of when using this: there shouldn't be done any schema changes while the whitelist/blacklist changes are processed and the connector should only be shut down *after* that processing has happened and a single binlog reader is running. There is DBZ-1091 for addressing these issues. Support for whitelist/blacklist changes is disabled by default for the time being and should be considered an experimental feature until the implementation has stabilized.

    Description

      We found impossible to alter config of an existed connector.

      Requirement :
      – ability to enhance table.whitelist config property of an existed container with time.

      Steps we did :

      1. create new connector with only one table presented in table.whitelist property
      2. verify initial snapshot has been taken
      3. verify new changes were read properly from binlog
      4. PUT new config using PUT /connectors/(string: name)/config standard kafka-connect method. Config was exactly the same as we used at first step with just new table addition to table.whitelist property
      5. we noticed that config update caused graceful restart/rebalance of all existed tasks. So, the expectation was that changes were applied immediately after all tasks started again.

      Existed Behaviour :
      – nothing changed : topic wasn't created for newly added table and initial snapshot haven't been made for it

      Expected Behavior :
      – new addition to table.whitelist property had to cause initial snapshot read for new tables with appropriate topic creation and data population from binlog

      Attachments

        Issue Links

          Activity

            People

              moirat Moira Tagle (Inactive)
              anton_nazaruk Anton Nazaruk (Inactive)
              Votes:
              8 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: