-
Bug
-
Resolution: Not a Bug
-
Blocker
-
None
-
Logging 6.1.0
-
False
-
None
-
False
-
NEW
-
OBSDA-849 - Enable Full In-Cluster OpenTelemetry Support In OpenShift Logging
-
NEW
-
-
Known Issue
-
Proposed
-
-
Description: Prune filter does not prune OTEL attributes on log streams.
CLF:
spec:
filters:
- name: prune-logs
prune:
in:
- .k8s.container.name
type: prune
- name: my-custom-labels
openshiftLabels:
custom-qe-labels-added: 'true'
type: openshiftLabels
managementState: Managed
outputs:
- lokiStack:
authentication:
token:
from: serviceAccount
dataModel: Otel
target:
name: lokistack-dev
namespace: openshift-logging
tuning:
compression: none
name: lokistack-otlp
tls:
ca:
configMapName: openshift-service-ca.crt
key: service-ca.crt
type: lokiStack
pipelines:
- filterRefs:
- my-custom-labels
- prune-logs
inputRefs:
- application
- infrastructure
- audit
name: my-pipeline
outputRefs:
- lokistack-otlp
serviceAccount:
name: loki-otlp
Here, we expect the OTEL attribute k8s.container.name to be pruned on log streams but it still continues to exist on logs.
Logs:
$ logcli --tls-skip-verify --bearer-token="$(oc whoami -t)" --addr "https://lokistack-dev-openshift-logging.apps.kbharti1029x.qe.devcluster.openshift.com/api/logs/v1/application" query '{k8s_namespace_name="my-app", k8s_container_name="centos-logtest"}' | tail -1 2024/10/29 14:28:07 https://lokistack-dev-openshift-logging.apps.kbharti1029x.qe.devcluster.openshift.com/api/logs/v1/application/loki/api/v1/query_range?direction=BACKWARD&end=1730226487909182000&limit=30&query=%7Bk8s_namespace_name%3D%22my-app%22%2C+k8s_container_name%3D%22centos-logtest%22%7D&start=1730222887909182000 2024/10/29 14:28:08 Common labels: {detected_level="debug", k8s_container_name="centos-logtest", k8s_namespace_name="my-app", k8s_node_name="ip-10-0-3-101.us-east-2.compute.internal", k8s_pod_label_fix_env_qe_ispresent="true", k8s_pod_name="centos-logtest-chx2t", k8s_pod_uid="2635a7ff-476c-4d3d-94a3-fb88b52f2652", kubernetes_container_name="centos-logtest", kubernetes_host="ip-10-0-3-101.us-east-2.compute.internal", kubernetes_namespace_name="my-app", kubernetes_pod_name="centos-logtest-chx2t", log_iostream="stdout", log_source="container", log_type="application", openshift_cluster_id="016174f6-2bb9-4cf0-aa7d-33531936f240", openshift_cluster_uid="016174f6-2bb9-4cf0-aa7d-33531936f240", openshift_label_custom_qe_labels_added="true", openshift_log_source="container", openshift_log_type="application", severity_text="default"} 2024-10-29T14:24:49-04:00 {observed_timestamp="1730226289705581596"} {"message": "MERGE_JSON_LOG=true", "level": "debug","Layer1": "layer1 0", "layer2": {"name":"Layer2 1", "tips":"Decide by PRESERVE_JSON_LOG"} , "StringNumber":"10", "Number": 10,"foo.bar":"Dot Item","{foobar}":"Brace Item","[foobar]":"Bracket Item", "foo:bar":"Colon Item","foo bar":"Space Item" }
vector.toml
.... [transforms.pipeline_my_pipeline_1_prune_logs_3] type = "remap" inputs = ["pipeline_my_pipeline_1_my_custom_labels_2"] source = ''' in = [["k8s","container","name"]] # Remove keys from in list for_each(in) -> |_index, val| { . = remove!(., val) } ''' ...
Version: Logging 6.1
Steps to reproduce:
1) Deploy CLO and LO v6.1
2) Create a CLF with a prune filter on one of the labels.
3) Observe that the attribute is pruned.
Actual Result: Logs still contain the pruned field
Expected Result: Logs should not contain the attribute which is being pruned on filter.
Additional Info: Using the viaq attribute 'kubernetes.pod_name', we can see the field being pruned.
- relates to
-
OBSDOCS-1468 Add note about OTEL limitations
- New