-
Feature
-
Resolution: Unresolved
-
Normal
-
None
-
Logging 6.1
-
None
-
False
-
None
-
False
-
Not Selected
-
0
-
Important
Proposed title of this feature request
Transformation and restructing of fields in Log Forwarder
What is the nature and description of the request?
Log receivers at customers often have specific requirements for how logs should be received. This request is asking for a feature that allows a remapping / transform of fields in log messages. This would allow customers to define their own fields / schema for log entries and to customise the fields that are sent to the receiver.
One specific use case would be when customers would like to use the Splunk HEC as a log receiver. The Splunk HEC endpoint has specific keys for `event` and `fields`, where an audit log would look like this:
{ "time": 1426279439, //convert requestReceivedTimestamp into epoch time "host": "example-master-2", //hostname "source": "http:openshift-example", //has to be a static value set by cluster-admin "sourcetype": "httpevent", //has to be a static value set by cluster-admin "index": "example_index", //has to be a static value set by cluster-admin "event": "/api/v1/namespaces/testing/secrets?allowWatchBookmarks=true", //contains "RequestURI" for KubernetesAPI audit logs or "message" for linux audit logs "fields": //contains the whole audit log { "annotations_authorization.k8s.io/decision": "allow", "annotations_authorization.k8s.io/reason": "", "apiVersion": "audit.k8s.io/v1", "auditID": "6b78966f-b699-4965-acc8-xxxxxxxxxxxx", "hostname": "example-master-2", "k8s_audit_level": "Metadata", "kind": "Event", "level": "Metadata", "log_source": "kubeAPI", "log_type": "audit", "objectRef_apiVersion": "v1", [..] "userAgent": "kubelet/v1.29.10+67d3387 (linux/amd64) kubernetes/2fa4795", "verb": "watch" } }
Vector allows this kind of remapping using VRL: https://vector.dev/docs/reference/configuration/transforms/remap/
So the feature in this RFE would allow customers to define arbitrary VRL to transform the log messages to the expected format.
Why does the customer need this? (List the business requirements)
Splunk licenses apply to the `event` field, but not to the `fields` field. The customer would be able to transform the log message into a format that would result in lower Splunk costs.
List any affected packages or components.
Log Collector (Vector)
Log Forwarding