Uploaded image for project: 'Managed Service - Streams'
  1. Managed Service - Streams
  2. MGDSTRM-10135

Add policy support to upstream quota plug-in by providing min available ratio support

XMLWordPrintable

    • False
    • None
    • False
    • No
    • MGDSRVS-164 - Provide Configuration Options for Kafka Instances
    • ---
    • ---
    • MK - Sprint 232

      WHAT

      Add quota policy support to the upstream quota plug-in.

      WHY

      The midstream version of the quota plug-in has a clear separation of concerns between various component for managing quotas and provides a much cleaner API for adding new policies and working with multiple log directories. Having this separation in the upstream quota plug-in will make adding KIP-827 support much cleaner and simpler. This distinction is outlined in Proposal 47.

      HOW

      This card aims to add an additional `ThrottleFactorPolicy` to apply the throttle when the available percentage of the disk drops below the thereshold.

      Specifically this card aims to use the `VolumeSource` and `ThrottleFactorPolicy` introduced by MGDSTRM-10135

      Its worth noting the naming of things evolved quite a bit during the proposal process. So the mappings are as follows:

      Midstream proposal upstream
      Limit Policy N/A N/A [1]
      ConsumedBytesLimitPolicy   TotalConsumedThrottleFactorPolicy
      MinFreeBytesLimitPolicy availableBytes AvailableBytesThrottleFactorPolicy
      MinFreePercentageLimitPolicy availableRatio TBC by this card
      QuotaFactorSupplier Throttle Factor Source ThrottleFactorPolicy
      DataSourceTask VolumeSource VolumeSource

      [1] The proposal got rid of the idea of soft limits as being of very limited value and thus the limit policies became much simpler. Without the need for composition. I.e a soft limit based on a percentage and a hard limit based on the absolute values.

              rh-ee-robeyoun Robert Young
              sbarker@redhat.com Sam Barker
              Kafka Integrations
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: