Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-843

Assemble multi-line exception stacktrace log messages



    • Multiline Stacktraces
    • 7
    • Done
    • OBSDA-113 - Support aggregating back multi-line log messages into a single log message
    • OBSDA-113Support aggregating back multi-line log messages into a single log message
    • 100
    • 100% 100%



      • Assemble log messages again that are part of a stacktrace to store it as one single log record instead of multiple.
      • Look at Java stacktraces at a minimum as this seems to be the most common language used for workloads deployed on OpenShift.


      • Additionally programming languages like Python and Go.


      Log messages as part of a stacktrace are often split into multiple lines where each line represents a new log record. A log collector takes every log record and stores it as an individual entry into a log storage. In this case, a user has no way to look at the single stacktrace or manual assemble it inside the log store and the context is completely lost which makes it harder to investigate the issue. Therefore, a log solution needs to store all lines as part of a stacktraces.


      Users can log stacktraces as JSON which would force the entire log to be a single log record instead of multiple per log line.

      Acceptance Criteria

      • Verify that a Java stacktrace is stored as a single log record.

      Risk and Assumptions

      Additional Notes

      • As in Christian's comment below[1], there is a Fluentd plugin from Google to detect stacktraces from different language[2].
      • Christian was successfully able to use this with a minimal Fluentd configuration (see[3]).
      • Found another example from someone who modified our Fluentd to support multiline. See https://gist.github.com/jgrumboe/e56d0ceb51026a5f39c576181a3bda23.
      • Splunk Connect for K8s multiline configuration[4]

      [1] https://issues.redhat.com/browse/LOG-843?focusedCommentId=14376072&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14376072
      [2] https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions
      [3] https://github.com/sichvoge/fluentd-detect-exception-example/blob/master/fluentd.conf
      [4] https://github.com/splunk/splunk-connect-for-kubernetes/blob/develop/manifests/splunk-kubernetes-logging/configMap.yaml#L148


        Issue Links



              jcantril@redhat.com Jeffrey Cantrill
              cvogel1 Christian Heidenreich (Inactive)
              Kabir Bharti Kabir Bharti
              6 Vote for this issue
              20 Start watching this issue