Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4022

Use with RDS destablizes DB with high perf impact

    XMLWordPrintable

Details

    • False
    • False

    Description

      Running Debezium with:

      • debezium/connect:1.6
      • Running in AWS ECS
      • PostgreSQL 12.7 on AWS RDS (db.m5.4xlarge / 16 vCPU / 64 GB RAM)
      • Kafka 2.7.0 via AWS MSK

      Within a few minutes of running, the load on RDS increases such that our DB becomes unusable and fails with a high number of errors (like pq: the database system is in recovery mode), forcing us to stop it to reduce load.

      RDS Perf insights lists the top wait offender as:
      SELECT COUNT(1) FROM pg_publication WHERE pubname = 'debezium_pub'

      This has been consistent with the past 2 versions we've tried, 1.5 and 1.6.

      Some logs, connector config, and RDS perf info is included as attachments.

      How can we reduce the perf impact?

      Attachments

        1. debezium-logs.txt
          117 kB
        2. jax-connector.json
          1 kB
        3. rds-cpu.png
          rds-cpu.png
          228 kB
        4. rds-freeable-memory.png
          rds-freeable-memory.png
          116 kB
        5. rds-load-by-waits.png
          rds-load-by-waits.png
          271 kB
        6. rds-monitor.png
          rds-monitor.png
          255 kB
        7. user-connector.json
          1 kB

        Activity

          People

            Unassigned Unassigned
            thnk2wn Geoff Hudik (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: