Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-3925

Collector in CrashLoop state when forwarding to Cloudwatch using fluentd

XMLWordPrintable

    • False
    • None
    • False
    • NEW
    • VERIFIED
    • Log Collection - Sprint 234

      Description: fluentd is in Error when forwarding to Cloudwatch on Log 5.6.5. Vector works fine.

      Error:

      2023-04-06 00:28:55 +0000 [error]: unexpected error error_class=RuntimeError error="Unable to find a compatible xml library. Ensure that you have installed or added to your Gemfile one of ox, oga, libxml, nokogiri or rexml"
      2023-04-06 00:28:55 +0000 [error]: /usr/local/share/gems/gems/aws-sdk-core-3.130.2/lib/aws-sdk-core/xml/parser.rb:74:in `set_default_engine'
      2023-04-06 00:28:55 +0000 [error]: /usr/local/share/gems/gems/aws-sdk-core-3.130.2/lib/aws-sdk-core/xml/parser.rb:96:in `<class:Parser>'
      2023-04-06 00:28:55 +0000 [error]: /usr/local/share/gems/gems/aws-sdk-core-3.130.2/lib/aws-sdk-core/xml/parser.rb:7:in `<module:Xml>'
      2023-04-06 00:28:55 +0000 [error]: /usr/local/share/gems/gems/aws-sdk-core-3.130.2/lib/aws-sdk-core/xml/parser.rb:5:in `<module:Aws>'
      2023-04-06 00:28:55 +0000 [error]: /usr/local/share/gems/gems/aws-sdk-core-3.130.2/lib/aws-sdk-core/xml/parser.rb:3:in `<top (required)>'
      2023-04-06 00:28:55 +0000 [error]: /usr/local/share/gems/gems/aws-sdk-core-3.130.2/lib/aws-sdk-core/xml.rb:8:in `require_relative'
      2023-04-06 00:28:55 +0000 [error]: /usr/local/share/gems/gems/aws-sdk-core-3.130.2/lib/aws-sdk-core/xml.rb:8:in `<top (required)>'
      2023-04-06 00:28:55 +0000 [error]: /usr/local/share/gems/gems/aws-sdk-core-3.130.2/lib/aws-sdk-core.rb:68:in `require_relative'
      .......
      $ oc get pods -n openshift-logging
      NAME                                           READY   STATUS      RESTARTS      AGE
      cluster-logging-operator-5574d47879-bhzbk      1/1     Running     0             10m
      collector-7vs5t                                1/2     Error       1 (8s ago)    16s
      collector-9dqx6                                1/2     Error       1 (11s ago)   17s
      collector-dj7xd                                1/2     Error       1 (11s ago)   17s
      collector-dn72q                                1/2     Error       1 (11s ago)   17s
      collector-lr9tz                                1/2     Error       1 (12s ago)   17s
      collector-xrwjx                                1/2     Error       1 (11s ago)   16s
      elasticsearch-cdm-lwfqfeuy-1-7f798dd4f-q7tkj   2/2     Running     0             10m
      elasticsearch-im-app-28012395-fjfg4            0/1     Completed   0             2m8s
      elasticsearch-im-audit-28012395-889bf          0/1     Completed   0             2m8s
      elasticsearch-im-infra-28012395-n78pm          0/1     Completed   0             2m8s
      kibana-7b8b94dd99-ltg6k                        2/2     Running     0             10m

      CSV:

      $ oc get csv -n openshift-logging
      NAME                            DISPLAY                            VERSION   REPLACES   PHASE
      cluster-logging.v5.6.5          Red Hat OpenShift Logging          5.6.5                Succeeded
      elasticsearch-operator.v5.6.5   OpenShift Elasticsearch Operator   5.6.5                Succeeded
      loki-operator.v5.6.5            Loki Operator                      5.6.5                Succeeded

      Steps to reproduce:
      1) Deploy CLO and EO v5.6.5
      2) Create CW secret and CLF for forwarding to Cloudwatch (CLF: http://pastebin.test.redhat.com/1096835)
      3) Create CL instance (CL: http://pastebin.test.redhat.com/1096837)

      CLO image: registry.redhat.io/openshift-logging/cluster-logging-rhel8-operator@sha256:36ca7a82023be7ac66255af586efa79a6056f6e161f5df90d9676a8ebb3b360e

      How reproducible : always

      Expected Result: Fluentd should forward logs to CW.

      Actual Result: Fluentd is in error

            vparfono Vitalii Parfonov
            rhn-support-kbharti Kabir Bharti
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: