-
Epic
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
None
-
Use OTLP/http for sending logs to the LokiStack 6.1
-
1
-
False
-
None
-
False
-
Not Selected
-
To Do
The logging OTLP model https://github.com/rhobs/observability-data-model/blob/main/cluster-logging.md
LokiStack OTLP docs https://loki-operator.dev/docs/open-telemetry.md/
Acceptance criteria:
- change docs https://docs.openshift.com/container-platform/4.17/observability/otel/otel-forwarding-telemetry-data.html#otel-forwarding-logs-to-tempostack_otel-forwarding-telemetry-data
- e2e test
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: otel-collector-logs-writer rules: - apiGroups: - loki.grafana.com resourceNames: - logs resources: - application verbs: - create - apiGroups: [""] resources: ["pods", "namespaces", "nodes"] verbs: ["get", "watch", "list"] - apiGroups: ["apps"] resources: ["replicasets"] verbs: ["get", "list", "watch"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: otel-collector-logs-writer roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: otel-collector-logs-writer subjects: - kind: ServiceAccount name: otel-collector namespace: openshift-logging --- apiVersion: opentelemetry.io/v1beta1 kind: OpenTelemetryCollector metadata: name: otel namespace: openshift-logging spec: config: extensions: bearertokenauth: filename: "/var/run/secrets/kubernetes.io/serviceaccount/token" receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: k8sattributes: {} resource: attributes: - key: kubernetes.namespace_name from_attribute: k8s.namespace.name action: upsert - key: kubernetes.pod_name from_attribute: k8s.pod.name action: upsert - key: kubernetes.container_name from_attribute: k8s.container.name action: upsert - key: log_type value: application action: upsert exporters: otlphttp: endpoint: https://logging-loki-gateway-http.openshift-logging.svc.cluster.local:8080/api/logs/v1/application/otlp encoding: json tls: ca_file: "/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt" auth: authenticator: bearertokenauth service: extensions: [bearertokenauth] pipelines: logs: receivers: [otlp] processors: [k8sattributes, resource] exporters: [debug,otlphttp]
LokiStack creates a following config for loki
otlp_config:
resource_attributes:
attributes_config:
- action: index_label
attributes:
- k8s.container.name
- k8s.cronjob.name
- k8s.daemonset.name
- k8s.deployment.name
- k8s.job.name
- k8s.namespace.name
- k8s.node.name
- k8s.pod.name
- k8s.statefulset.name
- kubernetes.container_name
- kubernetes.host
- kubernetes.namespace_name
- kubernetes.pod_name
- log_source
- log_type
- openshift.cluster.uid
- openshift.log.source
- openshift.log.type
- service.name
- action: structured_metadata
attributes:
- k8s.node.uid
- k8s.pod.uid
- k8s.replicaset.name
- process.command_line
- process.executable.name
- process.executable.path
- process.pid
- action: structured_metadata
regex: k8s\.pod\.labels\..+
- action: structured_metadata
regex: openshift\.labels\..+
log_attributes:
- action: structured_metadata
attributes:
- k8s.event.level
- k8s.event.object_ref.api.group
- k8s.event.object_ref.api.version
- k8s.event.object_ref.name
- k8s.event.object_ref.resource
- k8s.event.request.uri
- k8s.event.response.code
- k8s.event.stage
- k8s.event.user_agent
- k8s.user.groups
- k8s.user.username
- log.iostream
- action: structured_metadata
regex: k8s\.event\.annotations\..+
- action: structured_metadata
regex: systemd\.t\..+
- action: structured_metadata
regex: systemd\.u\..+
allow_structured_metadata: true