Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-940

Snapshot in parallel mode with PostgreSQL Debezium Connector

    XMLWordPrintable

Details

    • Feature Request
    • Resolution: Duplicate
    • Optional
    • None
    • 0.9.0.Alpha2
    • postgresql-connector
    • None
    • 0
    • 0% 0%

    Description

      Currently the snapshot with PostgreSQL connector is taken serially such as , if there are `TableA` and `TableB` with millions of records , and both the tables are either whitelisted by setting the property 'table.whitelist' as "schemaName.tableA,schemaName.tableB" OR 'table.whitelist' property is not set and default is taken , then snapshot will be taken in a way that first all processing is done wrt to tableA and then move to tableB , which gives a window of unavailability of data for tableB as topic in Kafka as it will be only be available after tableA processing has been finished in a serial fashion.
      The definition of done will be to take the snapshot in parallel fashion so the tables doesnot have to wait for another table to get finished before its data becomes available in Kafka topics.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              smsufyian Muhammad Sufyian (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: