Summary

On ACS Cloud Service, we require long-term AWS S3 storage.

It's particularly relevant for the type of logs where we have the following requirements:

• large log records (CloudWatch Logs has a limit for records of 256 KB). OpenShift (OSD) cluster audit logs exceed this size sometimes, and they are rejected
• a high amount of logs (CloudWatch Logs ingestion is expensive)
• long retention period (CloudWatch Logs storage costs are higher in comparison to S3; besides that, S3 offers tiering, which can additionally reduce costs)
• low access rate (i.e., audit logs are rarely accessed, but they are required for different compliances and also sometimes are required for investigation)

Acceptance Criteria

• API implemented in code
• Updated reference documentation
• Updated upstream examples and how-to documentation
• AWS S3 is supported output in Forwarder
  • set bucket + region
  • (nice to have) template for prefix key based on structured log field
• Forwarder to AWS S3 supports AWS STS authorization

Notes

Both collectors (Fluentd and Vector) support AWS S3 as output.

AWS STS authorization should be already supported since it's available for CloudWatch Logs.