-
Task
-
Resolution: Done
-
Major
-
None
-
5
-
False
-
-
False
-
NEW
-
NEW
-
This enhancement adds support for multiple CloudWatch outputs, each capable of using a distinct STS role for authentication.
-
Enhancement
-
-
-
3
-
Log Collection - Sprint 269
Summary
Update the operator code to deploy a ClusterLogForwarder to authenticate with AWS using a credentials file
so that it no longer utilizes environment variables.
Acceptance Criteria
- Verify the collector can still forward logs using single output to cloudwatch using an STS role.
- Verify the collector can forward to multiple AWS outputs using different(distinct) role ARNs.
- Verify the collecor can still forward to cloudwatch using static, long-lived credentials.
Notes
Following our arch meeting Mar 4th, it was decided that option2 "NO Change" to the API, is the preferred choice for feature implementation. This means we will parse the existing role_arn from the secret specified, and create our own "credentials" file secret, populating the vector config with a 'profile' that matches the sink name. So the path would be "/var/run/ocp-collector/credentials" or similar, and the auth.profile would be "output_my_cw" in this example.
Vector Configuration
[sinks.output_my_cw] type = "aws_cloudwatch_logs" inputs = ["output_my_cw_group_name"] region = "us-east-1" auth.credentials_file = "/var/run/ocp-collector/credentials" auth.profile = "output_my_cw"
AWS Credentials File - created as a Secret by CLO
[output_my_cw] web_identity_token_file=/var/run/ocp-collector/serviceaccount/token role_arn=arn:aws:iam::2612345066:role/test-namespace-role-for-sts role_session_name=output-my-cw
Note: If we are using the name of the output as the profile, do we need to cleanse or otherwise filter the name before using in the credentials file??
Proposed API Changes
A new credFile authentication type is introduced allowing the user to specify configMapName, key and profile:
cloudwatch:
authentication:
type: credFile
credFile:
configMapName: aws-creds
key: credentials
profile: default
Implementation Options:
1. ConfigMap-Based Credentials (shown above)
-
Users create a ConfigMap and key, containing the AWS credentials file.The file is mounted to a predefined root path ending with the key name (e.g., /root/.aws/credentials).Vector is configured to reference the credentials file and profileAdd error handling and/or validation of the file and contents.
2. Dynamic Credentials File Creation (not shown)
-
- Users specify IAM role details in the OpenShift Logging configuration using the existing iamRole and secretName/key.
- The operator generates an AWS credentials file and SA token dynamically (either separate files with profile [default] or all in one file with specified profiles)
- Vector is updated to use the generated credentials file.
- Error handling is added to validate the credentials file and contents
- This would NOT require an update to the API, but we would need to potentially invalidate the config if they are not using exactly as we expect?
Configuration Examples
Local AWS Credentials File
[default] web_identity_token_file=/var/run/ocp-collector/serviceaccount/token role_arn=arn:aws:iam::2612345066:role/test-namespace-role-for-sts role_session_name=ocp-default
Create ConfigMap from file
oc create configmap aws-creds --from-file=credentials=hack/my_aws_creds
Vector Configuration
[sinks.output_my_cw] type = "aws_cloudwatch_logs" inputs = ["output_my_cw_group_name"] region = "us-east-1" auth.credentials_file = "/root/.aws/credentials" auth.profile = "default"