-
Bug
-
Resolution: Won't Do
-
Normal
-
Logging 5.8.19, Logging 5.9.13
-
Future Sustainability
-
False
-
-
False
-
NEW
-
Release Notes, Troubleshoot
-
NEW
-
Known Issue
-
-
-
Moderate
Description of problem:
Not consistent the format between Fluentd and Vector when log forwarding to a syslog output where in Fluentd the message is without quotes and in Vector is with quotes.
Original log line as produced by application:
{"log_type":"openshift_audit","event type":"Sign on Success","userName":"XXXXX","event":"LEEF:1.0|redhat|openshift|4.12|authenticate devTime=2025-04-03T07:46:30.621410481-05:00[America/Chicago] devTimeFormat=yyyy-MM-dd HH:mm:ssZ requestClientApplication=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36 requestMethod=GET sourceServiceName=xxx.xxx.xx.xxx src=xxx.xxx.xx.xxx srcPort=8080 dst=xxx.xxx.xx.xxx dstPort=35302 proto=HTTP/1.1 apiUrl=https://server.example.com/resource-server/actuator/health"}
Log sent by Fluentd as received in the syslog server where the message doesn't contain quotes:
Apr 7 14:45:25 server.example.com fluentd {"@timestamp":"2025-04-07T14:45:24.699132308+00:00","message":{"log_type":"openshift_audit","event type":"Sign on Success","userName":"XXXXXXX","event":"LEEF:1.0|redhat|openshift|4.12|authenticate devTime=2025-04-03T07:46:30.621410481-05:00[America/Chicago] devTimeFormat=yyyy-MM-dd HH:mm:ssZ requestClientApplication=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36 requestMethod=GET sourceServiceName=xxx.xxx.xx.xxx src=xxx.xxx.xx.xxx srcPort=8080 dst=xxx.xxx.xx.xxx dstPort=35302 proto=HTTP/1.1 apiUrl=https://server.example.com/resource-server/actuator/health"},"docker":{"container_id":"6cfd642e5d466988ba6952844f539cd35ec1305908389fc1258b1550bae8ccb6"},"kubernetes":{"container_name":"agnhost","namespace_name":"syslogtest","pod_name":"hello-node-8dd54cb99-5hsbs","container_image":"registry.k8s.io/e2e-test-images/agnhost:2.43","container_image_id":"registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e","pod_id":"7076289c-ca8b-4e99-a7fc-fffe9f1e295e","pod_ip":"10.128.2.92","host":"server.example.com","labels":{"app":"hello-node","pod-template-hash":"8dd54cb99"},"master_url":"https://kubernetes.default.svc","namespace_id":"5b84ab56-c441-4942-b241-5e7a75799774","namespace_labels":{"kubernetes_io_metadata_name":"syslogtest","pod-security_kubernetes_io_audit":"restricted","pod-security_kubernetes_io_audit-version":"v1.24","pod-security_kubernetes_io_warn":"restricted","pod-security_kubernetes_io_warn-version":"v1.24"},"flat_labels":["app=hello-node","pod-template-hash=8dd54cb99"]},"level":"unknown","hostname":"server.example.com","pipeline_metadata":{"collector":{"ipaddr4":"10.37.205.135","inputname":"fluent-plugin-systemd","name":"fluentd","received_at":"2025-04-07T14:45:24.702567+00:00","version":"1.16.2 1.6.0"}},"openshift":{"sequence":131954,"cluster_id":"ec905b28-0bd2-4ab7-bcdc-201125e35249"},"viaq_msg_id":"NjI1NjU3OTItOGNmYi00ZDJmLWJiY2EtNmExZWI5MjY0NTc5","log_type":"application"}
Log sent by Vector as received in the syslog server where the message doesn't contain quotes:
Apr 7 14:45:24 server.example.com vector {"@timestamp":"2025-04-07T14:45:24.699132308Z","event":"LEEF:1.0|redhat|openshift|4.12|Apr 7 15:05:06 server.example.com vector {"@timestamp":"2025-04-07T15:05:06.245293157Z","event":"LEEF:1.0|redhat|openshift|4.12|authenticate devTime=2025-04-03T07:46:30.621410481-05:00[America/Chicago] devTimeFormat=yyyy-MM-dd HH:mm:ssZ requestClientApplication=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36 requestMethod=GET sourceServiceName=xxx.xxx.xx.xxx src=xxx.xxx.xx.xxx srcPort=8080 dst=xxx.xxx.xx.xxx dstPort=35302 proto=HTTP/1.1 apiUrl=https://server.example.com/resource-server/actuator/health","event type":"Sign on Success","file":"/var/log/pods/syslogtest_hello-node-8dd54cb99-5hsbs_7076289c-ca8b-4e99-a7fc-fffe9f1e295e/agnhost/0.log","hostname":"server.example.com","kubernetes":{"annotations":{"k8s.ovn.org/pod-networks":"{\"default\":{\"ip_addresses\":[\"10.128.2.92/23\"],\"mac_address\":\"0a:58:0a:80:02:5c\",\"gateway_ips\":[\"10.128.2.1\"],\"routes\":[{\"dest\":\"10.128.0.0/14\",\"nextHop\":\"10.128.2.1\"},{\"dest\":\"172.30.0.0/16\",\"nextHop\":\"10.128.2.1\"},{\"dest\":\"169.254.169.5/32\",\"nextHop\":\"10.128.2.1\"},{\"dest\":\"100.64.0.0/16\",\"nextHop\":\"10.128.2.1\"}],\"ip_address\":\"10.128.2.92/23\",\"gateway_ip\":\"10.128.2.1\"}}","k8s.v1.cni.cncf.io/network-status":"[{\n \"name\": \"ovn-kubernetes\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.128.2.92\"\n ],\n \"mac\": \"0a:58:0a:80:02:5c\",\n \"default\": true,\n \"dns\": {}\n}]","openshift.io/scc":"restricted-v2","seccomp.security.alpha.kubernetes.io/pod":"runtime/default"},"container_id":"cri-o://6cfd642e5d466988ba6952844f539cd35ec1305908389fc1258b1550bae8ccb6","container_image":"registry.k8s.io/e2e-test-images/agnhost:2.43","container_name":"agnhost","labels":{"app":"hello-node","pod-template-hash":"8dd54cb99"},"namespace_id":"5b84ab56-c441-4942-b241-5e7a75799774","namespace_labels":{"kubernetes_io_metadata_name":"syslogtest","pod-security_kubernetes_io_audit":"restricted","pod-security_kubernetes_io_audit-version":"v1.24","pod-security_kubernetes_io_warn":"restricted","pod-security_kubernetes_io_warn-version":"v1.24"},"namespace_name":"syslogtest","pod_id":"7076289c-ca8b-4e99-a7fc-fffe9f1e295e","pod_ip":"10.128.2.92","pod_name":"hello-node-8dd54cb99-5hsbs","pod_owner":"ReplicaSet/hello-node-8dd54cb99"},"level":"default","log_type":"openshift_audit","message":"{\"log_type\":\"openshift_audit\",\"event type\":\"Sign on Success\",\"userName\":\"XXXXXXX\",\"event\":\"LEEF:1.0|redhat|openshift|4.12|authenticate devTime=2025-04-03T07:46:30.621410481-05:00[America/Chicago] devTimeFormat=yyyy-MM-dd HH:mm:ssZ requestClientApplication=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36 requestMethod=GET sourceServiceName=xxx.xxx.xx.xxx src=xxx.xxx.xx.xxx srcPort=8080 dst=xxx.xxx.xx.xxx dstPort=35302 proto=HTTP/1.1 apiUrl=https://server.example.com/resource-server/actuator/health\"}","openshift":{"cluster_id":"ec905b28-0bd2-4ab7-bcdc-201125e35249","sequence":3273},"userName":"XXXXXXX"}
Version-Release number of selected component (if applicable):
Logging 5.8.19 and Logging 5.9.12
How reproducible:
Always
Steps to Reproduce:
- Deploy a syslog server
- Deploy the Cluster Logging Operator
- Configure clusterLogForwarder for log forwarding to a syslog server with the next configuration:
apiVersion: logging.openshift.io/v1 kind: ClusterLogForwarder metadata: name: instance namespace: openshift-logging spec: outputs: - name: logs type: syslog url: tcp://rsyslog-server.rsyslog-pj.svc:6514 pipelines: - inputRefs: - application - audit name: syslog-pl outputRefs: - logs
Actual results:
Differences in the format between Fluentd and Vector where the message is quoted in Vector and without quotes in Fluentd
Expected results:
Not differences in the format between Fluentd and Vector relative to be quoted the message or not.
Additional info:
This not expected change in the format of the message is not announced, then, when upgrading from Fluentd to Vector, it's broken some integrations with other tools for parsing the content.
