Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-6790

Update the operator cloudwatch forwarding to use credential file and profiles for authentication

XMLWordPrintable

    • 5
    • False
    • Hide

      None

      Show
      None
    • False
    • NEW
    • NEW
    • This enhancement adds support for multiple CloudWatch outputs, each capable of using a distinct STS role for authentication.
    • Enhancement
    • 3
    • Log Collection - Sprint 269

      Summary

      Update the operator code to deploy a ClusterLogForwarder to authenticate with AWS using a credentials file
      so that it no longer utilizes environment variables.

      Acceptance Criteria

      • Verify the collector can still forward logs using single output to cloudwatch using an STS role.​
      • Verify the collector can forward to multiple AWS outputs using different(distinct) role ARNs.​
      • Verify the collecor can still forward to cloudwatch using static, long-lived credentials.​

      Notes

      Following our arch meeting Mar 4th, it was decided that option2 "NO Change" to the API, is the preferred choice for feature implementation.   This means we will parse the existing role_arn from the secret specified, and create our own "credentials" file secret, populating the vector config with a 'profile' that matches the sink name.   So the path would be "/var/run/ocp-collector/credentials" or similar, and the auth.profile would be "output_my_cw" in this example.     

      Vector Configuration
      [sinks.output_my_cw]
      type = "aws_cloudwatch_logs"
      inputs = ["output_my_cw_group_name"]
      region = "us-east-1"
      auth.credentials_file = "/var/run/ocp-collector/credentials"
      auth.profile = "output_my_cw"
      AWS Credentials File - created as a Secret by CLO
      [output_my_cw]
      web_identity_token_file=/var/run/ocp-collector/serviceaccount/token
      role_arn=arn:aws:iam::2612345066:role/test-namespace-role-for-sts
      role_session_name=output-my-cw

      Note:  If we are using the name of the output as the profile, do we need to cleanse or otherwise filter the name before using in the credentials file??

       


      Proposed API Changes

      A new credFile authentication type is introduced allowing the user to specify configMapName, key and profile:

       

      cloudwatch:
        authentication:
          type: credFile
          credFile:
            configMapName: aws-creds
            key: credentials
            profile: default
      

       

      Implementation Options:

      1. ConfigMap-Based Credentials (shown above)

        • Users create a ConfigMap and key, containing the AWS credentials file.
        • The file is mounted to a predefined root path ending with the key name (e.g., /root/.aws/credentials).
        • Vector is configured to reference the credentials file and profile
        • Add error handling and/or validation of the file and contents.

       

      2. Dynamic Credentials File Creation (not shown)

        • Users specify IAM role details in the OpenShift Logging configuration using the existing iamRole and secretName/key.
        • The operator generates an AWS credentials file and SA token dynamically (either separate files with profile [default] or all in one file with specified profiles)
        • Vector is updated to use the generated credentials file.
        • Error handling is added to validate the credentials file and contents
        • This would NOT require an update to the API, but we would need to potentially invalidate the config if they are not using exactly as we expect?

      Configuration Examples

      Local AWS Credentials File
      [default]
      web_identity_token_file=/var/run/ocp-collector/serviceaccount/token
      role_arn=arn:aws:iam::2612345066:role/test-namespace-role-for-sts
      role_session_name=ocp-default
      
      Create ConfigMap from file
      oc create configmap aws-creds --from-file=credentials=hack/my_aws_creds
      Vector Configuration
      [sinks.output_my_cw]
      type = "aws_cloudwatch_logs"
      inputs = ["output_my_cw_group_name"]
      region = "us-east-1"
      auth.credentials_file = "/root/.aws/credentials"
      auth.profile = "default"
      

              rh-ee-calee Calvin Lee
              cahartma@redhat.com Casey Hartman
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: