Enhancement Proposal Story

As a cluster admin I want to be able to set a flow control policy (block, drop, ignore) and rate limits (bytes/sec) on selected containers:

a cluster-wide default for all containers (optional)
a special rate for containers in named namespaces
a special rate for containers matching a label selector

Rates are per-container and measured in bytes/sec, the default value 0 means no limits are applied.

Rates are enforced by dropping log data if it exceeds the rate.

Note: Enforcing rates using back-pressure will be addressed in a separate story. The API proposal should note how this will be added later.

Rate limits provide greater predictability and fairness in logging traffic.

Policy determines the quality of service for logging

drop: logs may be dropped if produced faster than consumed. This is sometimes called "at most once" delivery.
ignore: logs are ignored and will not be forwarded.

Acceptance Criteria

The design should

Cover the story outlined above
Note dependency and any extra requirements for the label selector proposal:
https://github.com/openshift/enhancements/blob/master/enhancements/cluster-logging/forwarder-label-selector.md
Be open to supporting new selectors that might be added to the ClusterLogForwarder
Be open to future enhancements that set shared limits for groups of containers where the total limit is shared by all members of the group.
Discuss fairness issues with rate 0.

The design should be amenable to implementation in stages:
1. Rates with drop policy, namespaces only
2. Block policy with/without rates limiting
*
Enhancement PR must be reviewed and accepted as usual.