Details
-
Bug
-
Resolution: Done
-
Undefined
-
Logging 5.4.0
-
False
-
False
-
NEW
-
OBSDA-108 - Distribute an alternate Vector Log Collector
-
VERIFIED
-
Logging (Core) - Sprint 213
Description
When forwarding logs to Loki using HTTPS and Vector as collector, getting the following error in Collector pod logs.
Jan 24 04:59:41.215 ERROR vector::cli: Configuration error. error=unknown field `username`, expected `user` or `password` for key `sinks.loki_server` at line 71 column 1 oc get pods NAME READY STATUS RESTARTS AGE cluster-logging-operator-784b7ff8d8-n78ls 1/1 Running 0 51m collector-84cql 1/2 CrashLoopBackOff 3 (19s ago) 63s collector-8vmc5 1/2 CrashLoopBackOff 3 (16s ago) 61s collector-brvpw 1/2 CrashLoopBackOff 3 (15s ago) 62s collector-pl5n9 1/2 CrashLoopBackOff 3 (23s ago) 62s collector-sw44q 1/2 Error 3 (41s ago) 63s collector-trq8r 1/2 Error 3 (40s ago) 62s elasticsearch-cdm-f7klylu7-1-587fbd4df5-msw59 2/2 Running 0 10m kibana-6ffcbd945d-l7bln 2/2 Running 0 10m
Steps to reproduce the isue:
1 Deploy Cluster Logging with Vector as Collector.
2 Create a secret for Loki HTTPS.
oc create secret generic loki-client -n openshift-logging --from-literal=user="USERNAME" --from-literal=password="API KEY"
3 Create a ClusterLogForwarder instance to forward logs to Loki.
apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
name: instance
namespace: openshift-logging
spec:
outputs:
- name: loki-server
type: loki
url: https://logs-prod3.grafana.net
secret:
name: loki-client
pipelines:
- name: to-loki
inputRefs:
- application
outputRefs:
- loki-server
4 Check the Collector pod are failing with error:
oc logs collector-pl5n9 -c collector Jan 24 05:04:47.486 INFO vector::app: Log level is enabled. level="debug" Jan 24 05:04:47.486 INFO vector::sources::host_metrics: PROCFS_ROOT is unset. Using default '/proc' for procfs root. Jan 24 05:04:47.486 INFO vector::sources::host_metrics: SYSFS_ROOT is unset. Using default '/sys' for sysfs root. Jan 24 05:04:47.487 INFO vector::app: Loading configs. path=[("/etc/vector/vector.toml", Some(Toml))] Jan 24 05:04:47.490 ERROR vector::cli: Configuration error. error=unknown field `username`, expected `user` or `password` for key `sinks.loki_server` at line 71 column 1
If we try to use the field user in the secret the CLF status is invalid, if we use the username the CLF is valid but the collector pods throws the above configuration error.
