Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-2812

[OCP 4.11] SSL Error on collector while sending logs to LokiStack as logstore

XMLWordPrintable

    • False
    • None
    • False
    • NEW
    • OBSDA-7 - Adopting Loki as an alternative to Elasticsearch to support more lightweight, easier to manage/operate storage scenarios
    • VERIFIED
    • Log Storage - Sprint 221, Log Storage - Sprint 222

      Error on fluentd pods:

      2022-07-11 19:49:11 +0000 [warn]: [loki_infra] failed to flush the buffer. retry_times=54 next_retry_time=2022-07-11 19:50:08 +0000 chunk="5e38918d52841a9c3eb9e6dc589aaab4" error_class=OpenSSL::SSL::SSLError error="SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)" 2022-07-11 19:49:11 +0000 [warn]: suppressed same stacktrace 2022-07-11 19:49:16 +0000 [warn]: [loki_audit] failed to flush the buffer. retry_times=53 next_retry_time=2022-07-11 19:50:24 +0000 chunk="5e389188d79e386fb00c9dcc982ae4a3" error_class=OpenSSL::SSL::SSLError error="SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)"

      CSV:

      [kbharti@cube ~]$ oc get csv
      NAME                    DISPLAY                     VERSION   REPLACES   PHASE
      clusterlogging.v5.5.0   Red Hat OpenShift Logging   5.5.0                Succeeded
      loki-operator.v5.5.0    Loki Operator               5.5.0                Succeeded
      

      Steps to reproduce:

      1) Deploy LokiOperator and LokiStack

      2) Create Cluster Logging instance with LokiStack as logstore

      spec:
        collection:
          logs:
            fluentd: {}
            type: fluentd
        logStore:
          lokistack:
            name: lokistack-dev
          type: lokistack
        managementState: Managed

      3) Create ClusterLogForwarder with logcollector-token as secret

      spec:
        outputs:
          - name: loki-app
            secret:
              name: logcollector-token-p2445
            type: loki
            url: >-
              https://lokistack-dev-gateway-http.openshift-logging.svc:8080/api/logs/v1/application
          - name: loki-infra
            secret:
              name: logcollector-token-p2445
            type: loki
            url: >-
              https://lokistack-dev-gateway-http.openshift-logging.svc:8080/api/logs/v1/infrastructure
          - name: loki-audit
            secret:
              name: logcollector-token-p2445
            type: loki
            url: >-
              https://lokistack-dev-gateway-http.openshift-logging.svc:8080/api/logs/v1/audit
        pipelines:
          - inputRefs:
              - application
            name: send-app-logs
            outputRefs:
              - loki-app
          - inputRefs:
              - infrastructure
            name: send-infra-logs
            outputRefs:
              - loki-infra
          - inputRefs:
              - audit
            name: send-audit-logs
            outputRefs:
              - loki-audit

      Expected Result: fluentd should be able to communicate with Loki and send logs for different tenants.

      Actual Result: SSL error on fluentd pods

              rojacob@redhat.com Robert Jacob
              rhn-support-kbharti Kabir Bharti
              Kabir Bharti Kabir Bharti
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: