Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6688

Debezium 2.3.0.Final Missing Kafka Channel Documentation

XMLWordPrintable

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Feature request or enhancement

      For feature requests or enhancements, provide this information, please:

      Which use case/requirement will be addressed by the proposed feature?

      Use of Kafka Channel for the PostgreSQL connector from version 2.3.0.Final.

      Implementation ideas (optional)

      I recently upgraded our Debezium connector to 2.3.0.Final explicitly to make use of the new Kafka channel.

      However, there are missing parts in the documentation, in my view, that made using the Kafka channel mare challenging.

      I would like to ask that you consider adding the following to the documentation:

      • it was not clear that both source and kafka channels need to be enabled: signal.enabled.channels: source,kafka . The original statement from the docs is "You can enable the Kafka signaling channel by adding it to the signal.enabled.channels configuration property". It is not clear what "it" stands for. Additionally, it is not explicitly stated that the proper configuration is to have source,kafka set. Rather, I double-checked with your IT tests to understand the set-up.
      • it is also not clear that Kafka channel cannot be enabled on its own for the PostrgeSQL connector since the signal table is still required for storing watermarks.
      • it is also not clear that the new logic creates its own Kafka Consumer. It would be, in my humble opinion, beneficial to explicitly state that and to maybe give an example of the core properties that need to be configured, namely the keystore and truststore properties of the consumer.
      • in the case of our deployment, we do not allow Debezium to create topics automatically. As such, it would be beneficial to state that in the case of manual topic configuration, the signal topic should have only a single partition
      • the consumer group ID (kafka-signal) is hard-coded in the code itself. This should also be stated in the docs, in my view, as it is not self-evident what it is. This also relates to the way we configure Kafka Connect for our use-case - we have created is a dedicated user with a limited permission scope to bolster security and we have to explicitly configure what consumer group IDs can be accessed via the Kafka Connect user.
      • it would be beneficial to have a Github example of how to set up the new Kafka channel for the PostgreSQL connector as well

            rh-ee-mvitale Mario Fiore Vitale
            mshtarbev Miroslav Shtarbev (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: