Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-8142

Add full support for a tags as a map of key-value pairs, for S3 and CloudWatch output types

XMLWordPrintable

    • Icon: Epic Epic
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • None
    • Log Collection
    • None
    • Support tags for AWS output types
    • Product / Portfolio Work
    • 5
    • False
    • Hide

      None

      Show
      None
    • False
    • NEW
    • NEW
    • 100% To Do, 0% In Progress, 0% Done
    • If Release Note Needed, Set a Value
    • Logging - Sprint 279, Logging - Sprint 280

      Goals

      Allow users to specify custom key-value tags via the ClusterLogForwarder (CLF) API. These tags are applied when the destination AWS resources (CloudWatch Log Groups or S3 Buckets) are first created by the log forwarder.

      Non-Goals

      No dynamic fields, only static strings.

      Motivation

      Key Benefits:

      • Cost Allocation: Tags enable accurate cost tracking and allocation across different teams or projects using AWS Billing and Cost Management tools.
      • Inventory and Governance: Tags can be used to organize and enforce policy across the logging infrastructure.

      Alternatives

      Acceptance Criteria

      • tags map can be added to the AWS output (both S3 and CloudWatch)
      • passes/fails validation when using  AWS output (both S3 and CloudWatch)
      • Generates vector remap config for AWS sink (both S3 and CloudWatch)
      • Tags are properly set at AWS (both S3 and CloudWatch)
      • unit tests to verify validation and config generation
      • mock e2e (not sure our mock library is up to date)??
      • Update feature documents
      • Update RH documentation

      Risk and Assumptions

      We assume the user is familiar with the errors resulting from incorrectly configuring tags.   If the tag passes our validation, but does not create the log group, the group should still be created and logs should not be dropped.

      Documentation Considerations

      links to AWS resources/docs

      Open Questions

      Additional Notes

      I have a POC branch that incorporates tags (as well as groupClass).   We want to make sure we use common methods for S3 and CW going forward.      This should be quick to parse the tags functionality:      
      https://github.com/cahartma/cluster-logging-operator/pull/4/files

              Unassigned Unassigned
              cahartma@redhat.com Casey Hartman
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: