Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1971

Offset storage contract should be independent of Kafka Connect

    XMLWordPrintable

Details

    Description

      Embedded engine and Debezium Server currently re-use the OffsetBackingStore contract from Kafka Connect. That can be confusing when implementing a custom store. We therefore should provide a Debezium-specific contract instead. This also applies to the configuration properties used e.g. to configure the file based store with the embedded engine.

      One specific requirement we should address is to avoid the usage of FQNs in the configuration. Instead, each implementation should expose a short name such as "file", "kafka", "dynamodb" etc. This would be retrieved from a store-specific factory which is discovered via the service loader. In the configuration then those short names woul be used by default, and the engine looks up the right implementation via the service loader. FQNs might still be supported as a fallback, should there be multiple implementations exposing the same short name.

      Attachments

        Issue Links

          is depended on by

          Feature Request - Feature requests from customers and/or users DBZ-6234 Separating Debezium from Kafka

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Open
          is related to

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) DBZ-1823 Provide a Debezium engine implementation based on framework classes

          • Critical - To be worked on immediately following BLOCKER issues.
          • Open
          relates to

          Feature Request - Feature requests from customers and/or users DBZ-3621 Saving Debezium states to JDBC database

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Feature Request - Feature requests from customers and/or users DBZ-7024 Create new implementation of DebeziumEngine

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Feature Request - Feature requests from customers and/or users DBZ-1807 Support converters for embedded engine

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Task - Represents a small unit of work that is not end-user facing. DBZ-3678 Provide Debezium-specific abstraction for offset storage SPI

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) DBZ-4509 Save and load offsets in Redis

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) DBZ-2292 Creating Offset Store using MySql database

          • Minor - To be worked after urgent, high, and medium priorities are resolved. This term is chosen when the severity of the issue is low, or the complexity or effort to correct it may be higher, relatively speaking. For low priority issues, known workarounds exist or are not needed due to the trivial effort needed to address the issue.
          • Open

          Activity

            People

              jpechane Jiri Pechanec
              gunnar.morling Gunnar Morling
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: