Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5181

Enable EmbeddedEngine to be used in a multi-threaded server

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Obsolete
    • Icon: Major Major
    • 2.4-backlog
    • None
    • embedded-engine
    • None

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Feature request or enhancement

      Which use case/requirement will be addressed by the proposed feature?

      In distributed databases like Yugabyte, tables are divided into multiple "tablets" and served by different servers. DML operations are redirected to the right tablet based on the sharing strategy. A CDC connector polls all tablets to collect CDC events.

      In this case, a multi-threaded implementation where each EmbeddedEngine is assigned a tablet improves throughput. Moreover, they can work independently of each other.

      However, the current implementation of the EmbeddedEngine assumed it will be called from a single thread. Specifically, it hard codes the number of taskConfigs to 0 and starts the task with the first taskConfig

       

      Also the EmbeddedEngine is a "public final" class and cannot be sub-classed.

      If these parameters are configurable with default of 0, then the EmbeddedEngine can be used in a multi-threaded server. Assigning independent tasks is the responsibility of the server and the connector to a distributed database. 

      Implementation ideas (optional)

              jcechace@redhat.com Jakub Čecháček
              vrajat Rajat Venkatesh (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: