Uploaded image for project: 'Managed Service - Streams'
  1. Managed Service - Streams
  2. MGDSTRM-9019

determine a better num.recovery.threads.per.data.dir value in kafka broker

XMLWordPrintable

    • MK - Sprint 225

      WHAT

      Kafka broker default the config `num.recovery.threads.per.data.dir` to 1, which means it will only have one thread to do log recovery, and let other CPUs idle. Because during log recovery, the broker basically does nothing, using only 1 thread for log recovery is wasteful and impact the performance.

      WHY

      Improve the log recovery performance 

      HOW

      Run benchmark tests to find out the best configuration for RHOSAK

      DONE

      1. Add the new `num.recovery.threads.per.data.dir` number as the default broker config in fleetshard
      2. Document the findings, ex: the performance gains

            lukchen@redhat.com Luke Chen
            lukchen@redhat.com Luke Chen
            Kafka Integrations
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: