WHAT

Add a mechanism to throttle message production using a configurable rate if there are any errors or inconsistencies while calculating the new throttle factor.

Note: we likely also want some concept of 'freshness' or 'snapshot' so that we can continue operating using an older throttle factor (or older observations) for some time in the presence of failure.

WHY

We intend to use KIP-827 to use the kafka admin api to describe disk usage of all log dirs. This introduces a variety of possible error conditions:

1. misconfigured admin client can't connect
2. transient network errors
3. timeouts
4. inconsistency between the active broker set between describing the cluster and describing the log dirs (using different api calls).

We want to make it configurable how to respond to these states. For example we could chose to fence the whole cluster by reducing the produce quota to effectively zero.

DONE

• PR merged upstream into kafka-quotas-plugins