Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5752

Redis Sink wait for Redis Replica writes

    XMLWordPrintable

Details

    • Feature Request
    • Resolution: Done
    • Major
    • 2.1.0.Alpha1
    • None
    • debezium-server
    • None
    • False
    • None
    • False
    • 0
    • 0% 0%

    Description

      Feature request or enhancement

      In case Redis  is configured with a replica shard, we should allow to verify that the data (XADD commands in the Sink and also history and offset) has been written to the replica. For that, these properties should be introduced (with the standard prefix):
      wait_enabled : Checks if the data has been written to the replica shard

      default: false

      wait_timeout: Timeout in milliseconds when checking write to the replica shard

      default: 1000 (minimum 1, as we don’t want to support 0 - blocking, as in case of an issue with the replica, everything will be stuck)

      retry_on_replica_failure: Ensures that the data has been written to the replica shard and keeps retrying if not

      default: true

      Implementation ideas (optional)

      Logic:

      If wait_enabled is true (should be false, by default), execute this wait command with 1 replica and the wait timeout

      If the result is not 1 (meaning, the replica shard has not been updated):

      if retry_on_replica_failure is false - just log an error and continue

      if true -sleep for X milliseconds (also should be a parameter) and try to write the same data again (do not proceed to the next batch).

      we use 1 as the number of replicas as in Redis Enterprise there could be only 1 replica shard.

      Attachments

        Activity

          People

            Unassigned Unassigned
            ggaborg Gabor Andras (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: