Summary

Update the operator code to deploy a ClusterLogForwarder to authenticate with AWS using a credentials file
so that it no longer utilizes environment variables.

Acceptance Criteria

• Verify the collector can still forward logs using single output to cloudwatch using an STS role.​
• Verify the collector can forward to multiple AWS outputs using different(distinct) role ARNs.​
• Verify the collecor can still forward to cloudwatch using static, long-lived credentials.​

Notes

Following our arch meeting Mar 4th, it was decided that option2 "NO Change" to the API, is the preferred choice for feature implementation.   This means we will parse the existing role_arn from the secret specified, and create our own "credentials" file secret, populating the vector config with a 'profile' that matches the sink name.   So the path would be "/var/run/ocp-collector/credentials" or similar, and the auth.profile would be "output_my_cw" in this example.     

Vector Configuration

[sinks.output_my_cw]
type = "aws_cloudwatch_logs"
inputs = ["output_my_cw_group_name"]
region = "us-east-1"
auth.credentials_file = "/var/run/ocp-collector/credentials"
auth.profile = "output_my_cw"

AWS Credentials File - created as a Secret by CLO

[output_my_cw]
web_identity_token_file=/var/run/ocp-collector/serviceaccount/token
role_arn=arn:aws:iam::2612345066:role/test-namespace-role-for-sts
role_session_name=output-my-cw

Note:  If we are using the name of the output as the profile, do we need to cleanse or otherwise filter the name before using in the credentials file??

Proposed API Changes

A new credFile authentication type is introduced allowing the user to specify configMapName, key and profile:

cloudwatch:
  authentication:
    type: credFile
    credFile:
      configMapName: aws-creds
      key: credentials
      profile: default

Implementation Options:

1. ConfigMap-Based Credentials (shown above)

• • Users create a ConfigMap and key, containing the AWS credentials file.
  • The file is mounted to a predefined root path ending with the key name (e.g., /root/.aws/credentials).
  • Vector is configured to reference the credentials file and profile
  • Add error handling and/or validation of the file and contents.

2. Dynamic Credentials File Creation (not shown)

• • Users specify IAM role details in the OpenShift Logging configuration using the existing iamRole and secretName/key.
  • The operator generates an AWS credentials file and SA token dynamically (either separate files with profile [default] or all in one file with specified profiles)
  • Vector is updated to use the generated credentials file.
  • Error handling is added to validate the credentials file and contents
  • This would NOT require an update to the API, but we would need to potentially invalidate the config if they are not using exactly as we expect?

Configuration Examples

Local AWS Credentials File

[default]
web_identity_token_file=/var/run/ocp-collector/serviceaccount/token
role_arn=arn:aws:iam::2612345066:role/test-namespace-role-for-sts
role_session_name=ocp-default

Create ConfigMap from file

oc create configmap aws-creds --from-file=credentials=hack/my_aws_creds

Vector Configuration

[sinks.output_my_cw]
type = "aws_cloudwatch_logs"
inputs = ["output_my_cw_group_name"]
region = "us-east-1"
auth.credentials_file = "/root/.aws/credentials"
auth.profile = "default"