Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2379

Implement new snapshot mode - initial_only

    XMLWordPrintable

Details

    Description

      I think, it is crucial to have an opportunity to reinitialize one or more tables from the whitelist.

      Let me explain, for example, I have many tables (which are indicated in the whitelist) in a database and the corresponding topic in Kafka.
      For one table with 50 000 records, a discrepancy of 50 entries between the table and the topic was noticed. Therefore, I would like to reinitialize the table.
      In total, there are more than 20 million records for all the tables from the whitelist, so reloading all these millions of records seems very expensive in terms of resources both from the database and Kafka sides.
      So, it would be great to run a separate connector with a new name and a config with, for example, snapshot.mode: initial_only, to have only a snapshot of the table that needs to be reinitialized,
      but leaving the topic name the same, so as to have the entire data in the original topic.

      At the moment there is no such mode and the connector in the snapshot.mode = initial, immediately after loading data from tables in the white list switches to load data from CDC tables.

      I'm sure, such feature would be very useful.

      Attachments

        Activity

          People

            ccranfor@redhat.com Chris Cranford
            disserakt Roman Ananyev (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: