Goals

Enhance the ClusterLogforwarder API to

• Allow tuning of individual output to support the unique characteristics of the receiver
• Reduce the possibility of log loss when the collector restarts
• Define a simple way to choose between throughput and durability of logs

Non-Goals

• Exposing the entirety of output configuration options for the underlying collector implementation

Motivation

• Some customers require collected log messages to survive a collector restart to support their regulatory mandates
• Some customers use outputs (e.g. Cloudwatch) that have hard limitations to the size of batches they can receive

Alternatives

Acceptance Criteria

• Verify their is no regression in log throughput and durability when the log forwarder does not spec any tuning
• Verify collected log messages are not lost when the output for a log forwarder is optimized for log durability

Risk and Assumptions

Documentation Considerations

• API documentation to support the added fields
• Usage documentation to explain the feature

Open Questions

• How does e2e acks compare to disk buffering

Additional Notes