Goals

Make modifications to the existing fluentd generated configuration to optimize the existing technology as much as possible. The overall intent is make changes without modifying the message payload where appropriate. There are several places where we may be able to remove or change keys that are not relevent to a particular output (e.g viaq_message_id for kafka)

Non-Goals

Motivation

We want to improve throughput performance as best as possible without making larger changes that might break customers

Alternatives

Alternatively we can do nothing and put our efforts into other collectors but those evaluations are still yet to be done.

Acceptance Criteria

• Clean, straight forward configuration that has minimal config to support the source pipeline
• Collector config is upgraded without requiring changes to outputs

Risk and Assumptions

Documentation Considerations

The changes should all be internal and unknown to the customers. May wish to document any performance improvements

Open Questions

Additional Notes