-
Feature
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
False
-
-
False
-
Not Selected
-
0
Proposed title of this feature request
Support for enabling the structured_metadata option in the Vector
What is the nature and description of the request?
Option to enable structured_metadata = true for vector (sink=loki) via CLF . Below option is there in Upstream https://vector.dev/docs/reference/configuration/sinks/loki/#structured_metadata
From loki end for version 6.3 and schema v13 below is enabled:
% oc get cm logging-loki-config -oyaml | grep -i structured allow_structured_metadata: true
Loki needs allow_structured_metadata: true to accept structured metadata. (which we have in 6.3)
But Vector needs structured_metadata = true to actually send it.
Why does the customer need this? (List the business requirements)
The need for structured metadata is as a replacement for sending labels for kubernetes-specific data (e.g. openshift pod name, resource owner, node etc.). Although necessary for effective querying, in loki a high number of changing labels is regarded as an anti-pattern and can lead to excessive object storage usage, degradation in loki's performance and very slow queries, as explained here: https://grafana.com/docs/loki/latest/get-started/labels/structured-metadata/
In order to build useful dashboards and alerts with openshift logs while avoiding the slowness and costs associated with a large number of dynamic labels (especially for large clusters, with hundreds of namespaces and thousands of short-lived pods) structured metadata is required.
List any affected packages or components.
Openshift Logging, Vector