Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7421

Poor performance with incremental snapshot with long list of tables

XMLWordPrintable

    • False
    • None
    • False

      Bug report

      What Debezium connector do you use and what version?

      Mysql Connector - Debezium 2.4.0

      What is the issue?

      When an incremental snapshot is running, and a lot of input binlog messages are being processed in parallel, the processing performance can become very poor.

      After profiling the application, this is correlated to the size of the "dataCollectionsToSnapshot" data structure. In a nutshell, the more tables you snapshot, the worse the performance will get.

      Profiling results:

      As seen in the results, this is due to the call to "dataCollectionsToSnapshotAsString" for each new binlog records in case a snapshot is running.

       

       

      This might explain issue DBZ-7256

            rh-ee-mvitale Mario Fiore Vitale
            maximethebault Maxime Thebault (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: