Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-1499

[release-5.2] Error "com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 start byte 0x92" in Elasticsearch/Fluentd logs

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Critical
    • Logging 5.2
    • Logging 5.0, Logging 5.1, openshift-4.6
    • Log Collection
    • False
    • False
    • NEW
    • NEW
    • Undefined
    • Logging (Core) - Sprint 203, Logging (Core) - Sprint 204

    Description

      It's possible to see in the Elasticsearch error messages like:

      ~~~
      021-04-20T07:10:45.091466957Z [2021-04-20T07:10:45,089][WARN ][r.suppressed ] [elasticsearch-cdm-j69zbfpm-1] path: /_bulk, params: {}
      2021-04-20T07:10:45.091466957Z com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 start byte 0x92
      2021-04-20T07:10:45.091466957Z at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@21cdf37b; line: 1, column: 81]
      2021-04-20T07:10:45.091466957Z at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1702) ~[jackson-core-2.8.11.jar:2.8.11]
      ~~~

      Similar error is possible to read on the fluentd side:

      ~~~
      fluentd-f29lv/fluentd/fluentd/logs/current.log:2021-04-19T18:20:29.886010127Z 2021-04-19 18:20:29 +0000 [warn]: [retry_clo_default_output_es] failed to flush the buffer. retry_time=1 next_retry_seconds=2021-04-19 18:20:30 +0000 chunk="5b83a8bc000a55be294f55337118ffa7" error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch cluster ({:host=>\"elasticsearch.openshift-logging.svc.cluster.local\", :port=>9200, :scheme=>\"https\", :user=>\"fluentd\", :password=>\"obfuscated\"}): [500] {\"error\":{\"root_cause\":[

      {\"type\":\"json_parse_exception\",\"reason\":\"Invalid UTF-8 start byte 0x92\\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@204fe9c9; line: 1, column: 81]\"}

      ],\"type\":\"json_parse_exception\",\"reason\":\"Invalid UTF-8 start byte 0x92
      n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@204fe9c9; line: 1, column: 81]\"},\"status\":500}"
      fluentd-f29lv/fluentd/fluentd/logs/current.log:2021-04-19T18:20:35.296756674Z 2021-04-19 18:20:35 +0000 [warn]: [retry_clo_default_output_es] failed to flush the buffer. retry_time=0 next_retry_seconds=2021-04-19 18:20:36 +0000 chunk="5b83a8bc000a55be294f55337118ffa7" error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch cluster ({:host=>\"elasticsearch.openshift-logging.svc.cluster.local\", :port=>9200, :scheme=>\"https\", :user=>\"fluentd\", :password=>\"obfuscated\"}): [500] {\"error\":{\"root_cause\":[

      {\"type\":\"json_parse_exception\",\"reason\":\"Invalid UTF-8 start byte 0x92\\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@4cfcdc0; line: 1, column: 81]\"}

      ],\"type\":\"json_parse_exception\",\"reason\":\"Invalid UTF-8 start byte 0x92
      n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@4cfcdc0; line: 1, column: 81]\"},\"status\":500}"

      Attachments

        Activity

          Public project attachment banner

            context keys: [headless, issue, helper, isAsynchronousRequest, project, action, user]
            current Project key: LOG

            People

              jcantril@redhat.com Jeffrey Cantrill
              jcantril@redhat.com Jeffrey Cantrill
              Qiaoling Tang Qiaoling Tang
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: