Details
-
Bug
-
Resolution: Done
-
Undefined
-
Logging 5.7.0
-
False
-
None
-
False
-
NEW
-
NEW
-
-
Log Collection - Sprint 233, Log Collection - Sprint 234
Description
The clo generate an invalid parameter 'encoding "application/x-ndjson"'. It should be content_type: "application/x-ndjson". refer to https://docs.fluentd.org/output/http
Step to reproduce:
1. Deploy CLO.
2. Create fluentd receiver.
3. Forward logs to fluentd receiver over http: { "apiVersion": "logging.openshift.io/v1", "kind": "ClusterLogForwarder", "metadata": { "name": "instance", "namespace": "openshift-logging" }, "spec": { "outputs": [ { "http": { "headers": { "h1": "v1", "h2": "v2" }, "method": "POST" }, "name": "httpout-app", "type": "http", "url": "http://fluentdserver-fluentdserver.apps.anli13.qe.devcluster.openshift.com/logs/app" }, { "http": { "headers": { "h1": "v1", "h2": "v2" }, "method": "POST" }, "name": "httpout-infra", "type": "http", "url": "http://fluentdserver-fluentdserver.apps.anli13.qe.devcluster.openshift.com/logs/infra" }, { "http": { "headers": { "h1": "v1", "h2": "v2" }, "method": "POST" }, "name": "httpout-audit", "type": "http", "url": "http://fluentdserver-fluentdserver.apps.anli13.qe.devcluster.openshift.com/logs/audit" } ], "pipelines": [ { "inputRefs": [ "application" ], "name": "app-logs", "outputRefs": [ "httpout-app" ] }, { "inputRefs": [ "infrastructure" ], "name": "infra-logs", "outputRefs": [ "httpout-infra" ] }, { "inputRefs": [ "audit" ], "name": "audit-logs", "outputRefs": [ "httpout-audit" ] } ] } }
4. Check the fluentd pod and fluent.conf
Actual result:
2023-03-13 08:37:42 +0000 [warn]: parameter 'encoding' in <match **>
@type http
endpoint "http://fluentdserver-fluentdserver.apps.anli13.qe.devcluster.openshift.com/logs/audit"
http_method post
encoding application/x-ndjson
headers
<buffer>
@type "file"
path "/var/lib/fluentd/httpout_audit"
flush_mode interval
flush_interval 1s
flush_thread_count 2
retry_type exponential_backoff
retry_wait 1s
retry_max_interval 60s
retry_timeout 60m
queued_chunks_limit_size 573
total_limit_size 4809148876
chunk_limit_size 8388608
overflow_action block
disable_chunk_backup true
</buffer>
</match> is not used.
<label @HTTPOUT_AUDIT> <match **> @type http endpoint http://fluentdserver-fluentdserver.apps.anli13.qe.devcluster.openshift.com/logs/audit http_method post encoding "application/x-ndjson" headers {"h1":"v1","h2":"v2"} <buffer> @type file path '/var/lib/fluentd/httpout_audit' flush_mode interval flush_interval 1s flush_thread_count 2 retry_type exponential_backoff retry_wait 1s retry_max_interval 60s retry_timeout 60m queued_chunks_limit_size "#{ENV['BUFFER_QUEUE_LIMIT'] || '32'}" total_limit_size "#{ENV['TOTAL_LIMIT_SIZE_PER_BUFFER'] || '8589934592'}" chunk_limit_size "#{ENV['BUFFER_SIZE_LIMIT'] || '8m'}" overflow_action block disable_chunk_backup true </buffer> </match>