Summary

As an administrator,
I want to include or exclude logs being collected using metadata
so that I collect only the logs I am interested in.

As a developer,
I want to include or exclude logs being collected using metadata
in order to reduce the memory and cpu load on the collector

Acceptance Criteria

• Verify logs are collected or excluded based upon pod namespace
• Verify logs are collected or excluded based upon matching pod labels
• Verify logs are collected or excluded based upon matching pod expressions
• Verify logs are collected or excluded based upon container name
• Verify namespace, excludeNamespaces, containers support the use of exact names
• Verify namespace, excludeNamespaces, containers support the use of path globs (e.g. foo*, b*r)
• Verify namespace, excludeNamespaces only allow kubernetes valid namespaces names plus '*'

Notes

Proposed Spec

spec:
  inputs:
  - name:
    application:
      namespaces: []              # exact or glob
      excludeNamespaces: []       # exact or glob
      selector:
        matchLabels:
        matchExpressions:
        - key:
          operator:
          values: []
      containers:
        include:      []string
        exclude:      []string        #support exact or glob

Original Notes

The ClusterLogForwarder already allows selecting of logs that belong to a given namespace:

inputs:
  name: interesting
  application:
    namespaces: [fun, exciting, wild] 

This story adds a new field excludeNamespaces

inputs:
  name: interesting
  application:
    namespaces: [fun, exciting, wild]
    excludeNamespaces: [dull, boring, tedious] 

This excludes logs from all namespaces on the list.

Semantics

• namepaces: include only logs from namespaces on the list.
• excludeNamespaces: include logs from all namespaces not on the list.
• both: include only logs from namespaces listed in  namespaces that are not in excludeNamespaces. In other words excludeNamespaces takes precedence.

Note: the namespace filters take precedence over other filters, logically we:

1. Exclude all logs that not selected by namespaces and excludeNamespaces.
2. Apply other filtering rules (labels etc.)