-
Feature
-
Resolution: Done
-
Undefined
-
None
-
None
Proposed title of this feature request
"Reliability and performance tuning for log collection"
What is the nature and description of the request?
- Give users more control over reliability and performance tuning parameters, while retaining a simpler configuration API than exposing the underlying collector.
- Make the vector collector robust to to log loss during short term collector restarts, on the order of seconds or minutes. For example restarts due to config changes or killing collector pods to be immediately restarted by the daemonset.
Why does the customer need this? (List the business requirements)
We have frequent customer requests for greater control over the reliability or throughtput of the collector. Customers often phrase this requirement as "we need buffering", but in fact what they want is reliability - to avoid losing logs. Buffering is one technique in our toolbox, but there are others. As long as we avoid log loss, customers will be happy however we do it.
This issue has been raised specifically as an obstruction to migrating to vector. We must solve that problem, but we need to do it under a coherent API that covers this range of issue.s
List any affected packages or components.
Affects the collector.
- blocks
-
LOG-4565 Enhance sink buffering to all dropping oldest messages
- To Do
- is documented by
-
OBSDOCS-808 [DOC] Reliability and performance tuning for log collection
- New
- is duplicated by
-
OBSDA-333 Allow configuration of persistent storage for file buffers for the Vector log collector
- Closed
- links to