Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-883

Expose ability to forward logs only from specific pods via a label selector inside the Log Forwarding API.

XMLWordPrintable

    • Forward based on pod labels
    • Done
    • 0% To Do, 0% In Progress, 100% Done
    • Hide
      * As a cluster administrator, you can use Kubernetes pod labels to gather log data from an application and send it to a specific log collector. You do this by configuring the `inputs[].name.application.selector.matchLabels` element in the ClusterLogForwarder custom resource (CR) YAML file. You can also filter that log data by namespace. (link:https://issues.redhat.com/browse/LOG-883[*LOG-883*])
      Show
      * As a cluster administrator, you can use Kubernetes pod labels to gather log data from an application and send it to a specific log collector. You do this by configuring the `inputs[].name.application.selector.matchLabels` element in the ClusterLogForwarder custom resource (CR) YAML file. You can also filter that log data by namespace. (link: https://issues.redhat.com/browse/LOG-883 [* LOG-883 *])

      Goals

      • Expose an additional configuration inside the ClusterLogForwarder CR that allows to define a list of pod labels (key/value pair).

      Non-Goals

      n/a

      Motivation

      The Log Forwarding API currently only exposes the ability for users to select logs by their namespace. Although that gives a lot of flexibility already, we've seen cases where more is needed. For example, there are some applications that may be deployed multiple times on multiple namespaces (e.g. any webserver). Now you only want aggregate all logs from this specific application to a single location (e.g. if you created dedicated indices in Elasticsearch or log groups on CloudWatch).

      If there was a way to select logs from only certain applications identified by pod labels, you could target specific indices or use dedicated systems.

      Alternatives

      The Log Forwarding API abstracts away the individual technology underneath and uses standard k8s features to identify applications of interest. The two standard ways to group and identify k8s applications are namespaces and labels. There are no other alternative k8s-native ways to identify or select groups of k8s objects.

      Note: K8s annotations are used for non-identifying metadata so are not suitable for this purpose.

      Acceptance Criteria

      • Verify that the receiving system only receives the selected logs.
      • Verify that using a combination of `namespaces` and the new option (e.g. `labels`) will filter logs based on both values (e.g. forward logs from pods in selected namespaces and with the defined labels).

      Risk and Assumptions

      n/a

      Documentation Considerations

      • Document the syntax and limitations for the new Log Forwarding API field.
      • Document behaviour to when users mix namespace and pod label filtering.
      • Provide an example.

      Previous Work (Optional)

      Engineering notes

            rhn-engineering-aconway Alan Conway
            cvogel1 Christian Heidenreich (Inactive)
            Kabir Bharti Kabir Bharti
            Votes:
            3 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: