Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-2379

[release-5.4] Allow users to tune fluentd

    XMLWordPrintable

Details

    • False
    • False
    • NEW
    • OBSDA-53 - Add support for using Kerberos authentication for Kafka inside the Log Forwarding API
    • VERIFIED
    • Undefined
    • Before this update, tuning the buffer chunk size could cause the collector to generate a warning about the chunk size exceeding the byte limit for the event stream. With this update, you can also tune the read line limit, resolving the issue.
    • Logging (Core) - Sprint 216
    • No PX recommendation - schedule improvements, discuss merits, or close

    Description

      1. Proposed title of this feature request
        max_send_limit_bytes modifiable via API
      2. What is the nature and description of the request
        When the `buffer_chunk_size` in fluentd is configured the following warnings are written to the log file:
        2020-11-20 09:26:28 +0000 [warn]: chunk bytes limit exceeds for an emitted event stream: 1471470bytes
        

        This is caused by the `read_lines_limit` setting in the `in_tail` plugin of fluentd. To avoid the warning message it is advised to set the `read_lines_limit` to a smaller value as can be found here. However this variable can not be set via the settings of the openshift-logging operator. For customers who need to set `buffer_chunk_size` to match their external backend it would be a benefit to avoid unnecessary warnings if they were able to configure this setting as well. Furthermore customers are experiencing data loss because of this.

      1. List any affected packages or components.
        • fluentd

      Attachments

        Activity

          People

            jcantril@redhat.com Jeffrey Cantrill
            rhn-support-tmicheli Tobias Michelis
            Qiaoling Tang Qiaoling Tang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: