Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4478

Partition-scoped metrics for the SQL Server connector

XMLWordPrintable

      The connector task working with multiple partitions should expose metrics in a different format. The existing metrics logically can be divided into two categories:

      1. Task-scoped metrics. For instance, QueueTotalCapacity. While processing multiple source databases, the task will use a single queue for the messages sourced from all partitions.
      2. Partition-scoped metrics. For instance, MilliSecondsBehindSource. Such metrics should be reported individually for each source database which is currently impossible.

      The introduction of partition-scoped metrics is necessary for the following reasons:

      1. Being able to operate the connector. Without a partition identifier, the partition-scoped metrics from different partitions would clash and lose their meaning.
      2. Being able to write integration tests for multi-partition scenarios. The integration test suite synchronizes with the connector by means of metrics. For instance, there is a typical scenario where the connector waits for a snapshot to be completed. In a multi-partition configuration, the connector should either wait for a specific database snapshotted, or all databases snapshotted. It's impossible to implement on top of the currently existing metrics.

            Unassigned Unassigned
            sergeimorozov Sergei Morozov (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: