Goals

• A simple API for an specifying log forwarding to an S3 output
• A set of sane defaults for organizing log streams written to the user defined bucket
• The ability to modify the way log streams are organized when written to the user defined bucket
• Re-use existing AWS authentication features provided by the Cloudwatch output

Non-Goals

• To provide an API the exposes all the configuration points of the underlying collector implementation

Motivation

The primary motivation is to satisfy functionality requests from Red Hat managed services teams which are providing managed clusters for customers. They have requirements to be able to collect, forward, and store logs from both the hosted control plane and the management clusters utilizing credentials from multiple organizations in a cost efficient manor.

Alternatives

Acceptance Criteria

• Verify the log collector forwards logs to an S3 bucket
• Verify the S3 output supports assumeRole authentication from static keys
• Verify the S3 output supports assumeRole authentication from bearer token
• Verify the S3 output supports a template field for the s3 key prefix
• Verify the S3 output supports injected a formatted date field as part of the s3 key prefix
• Verify this feature is gated as tech-preview (dev preview)??
• Verify the log records stored in S3 are correctly grouped and organized according to the key prefix definition

Risk and Assumptions

Documentation Considerations

• Official feature documentation
• Upstream developer feature documentation
• API reference updates
• Release Notes

Open Questions

1. Do we need to support `filename_time_format` to address the key prefix functionality proposed by the draft PR
2. Is there a need to introduce this feature as tech-preview with a `v2beta1` API to allow the "soak" time for the API and additional testing?

Additional Notes