Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-2843

tls.cert, tls.key and passphrase are not passed to the fluentd configuration when forwarding logs using syslog over TLS

XMLWordPrintable

    • False
    • None
    • False
    • NEW
    • VERIFIED
    • Hide
      In previous releases of OpenShift Logging, when forwarding logs using syslog over TLS, the configuration for the fluentd log forwarder only contained the 'ca-bundle.crt' entry, but not the 'tls.crt', 'tls.key' or 'passphrase' entries, even if specified in the corresponding ClusterLogForwarder output. This issues has now been addressed and ca-bundle.crt, tls.crt, tls.key and passphrase are all passed to the configuration of the fluentd log forwarder.
      Show
      In previous releases of OpenShift Logging, when forwarding logs using syslog over TLS, the configuration for the fluentd log forwarder only contained the 'ca-bundle.crt' entry, but not the 'tls.crt', 'tls.key' or 'passphrase' entries, even if specified in the corresponding ClusterLogForwarder output. This issues has now been addressed and ca-bundle.crt, tls.crt, tls.key and passphrase are all passed to the configuration of the fluentd log forwarder.
    • Hide

      Deploy RHOL

      $ oc get csv -n openshift-logging
      NAME                           DISPLAY                            VERSION   REPLACES   PHASE
      cluster-logging.5.4.3          Red Hat OpenShift Logging          5.4.3                Succeeded
      elasticsearch-operator.5.4.3   OpenShift Elasticsearch Operator   5.4.3                Succeeded
       

      Create a secret containing `ca-bundle.crt`, `tls.crt` and `tls.key` for being used later in the clusterLogForwarder for sending to syslog:

      $ oc get secret syslog -o yaml 
      apiVersion: v1
      data:
        ca-bundle.crt: xxxxx
        tls.crt: xxxxx
        tls.key: xxxxx
      kind: Secret
      metadata:
        creationTimestamp: "2022-07-20T15:01:27Z"
        name: syslog
        namespace: openshift-logging
        resourceVersion: "7350054"
        uid: 5a4f01b3-e6f8-4f55-a284-9976217810a6
      type: Opaque

      Configure clusterLogForwarder for sending logs to rsyslog using tls 

      $ cat clusterlogforwarder.yaml
      apiVersion: logging.openshift.io/v1
      kind: ClusterLogForwarder
      metadata:
        labels:
          app.kubernetes.io/instance: logging
        name: instance
        namespace: openshift-logging
      spec:
        outputs:
          - name: syslog
            secret:
              name: syslog
            syslog:
              addLogSource: true
              facility: syslog
              rfc: RFC3164
              severity: informational
            type: syslog
            url: 'tls://localhost:514'
        pipelines:
          - inputRefs:
              - application
            name: application-logs
            outputRefs:
              - syslog
       $ oc create -f clusterlogforwarder.yaml

      Check the fluentd configuration generated that it only contains the `ca-bundle.crt` and not the `tls.key` and `tls.crt`:

      $ oc get cm collector -o json |jq '.data."fluent.conf"' -r
      ...
        <match journal.** system.var.log**>
          @type remote_syslog
          @id syslog_journal
          host localhost
          port 514
          rfc rfc3164
          facility syslog
          severity informational
          program ${$.systemd.u.SYSLOG_IDENTIFIER}
          protocol tcp
          packet_size 4096
          hostname "#{ENV['NODE_NAME']}"
          tls true
          ca_file '/var/run/ocp-collector/secrets/syslog/ca-bundle.crt'
          timeout 60
          timeout_exception true
          keep_alive true
          keep_alive_idle 75
          keep_alive_cnt 9
          keep_alive_intvl 7200
          <format>
            @type json
          </format>
          <buffer $.systemd.u.SYSLOG_IDENTIFIER>
            @type file
            path '/var/lib/fluentd/syslog_journal'
            flush_mode interval
            flush_interval 1s
            flush_thread_count 2
            retry_type exponential_backoff
            retry_wait 1s
            retry_max_interval 60s
            retry_timeout 60m
            queued_chunks_limit_size "#{ENV['BUFFER_QUEUE_LIMIT'] || '32'}"
            total_limit_size "#{ENV['TOTAL_LIMIT_SIZE_PER_BUFFER'] || '8589934592'}"
            chunk_limit_size "#{ENV['BUFFER_SIZE_LIMIT'] || '8m'}"
            overflow_action block
            disable_chunk_backup true
          </buffer>
        </match>  <match **>
          @type remote_syslog
          @id syslog
          host localhost
          port 514
          rfc rfc3164
          facility syslog
          severity informational
          protocol tcp
          packet_size 4096
          hostname "#{ENV['NODE_NAME']}"
          tls true
          ca_file '/var/run/ocp-collector/secrets/syslog/ca-bundle.crt'
          timeout 60
          timeout_exception true
          keep_alive true
          keep_alive_idle 75
          keep_alive_cnt 9
          keep_alive_intvl 7200
          <format>
            @type json
          </format>
          <buffer>
            @type file
            path '/var/lib/fluentd/syslog'
            flush_mode interval
            flush_interval 1s
            flush_thread_count 2
            retry_type exponential_backoff
            retry_wait 1s
            retry_max_interval 60s
            retry_timeout 60m
            queued_chunks_limit_size "#{ENV['BUFFER_QUEUE_LIMIT'] || '32'}"
            total_limit_size "#{ENV['TOTAL_LIMIT_SIZE_PER_BUFFER'] || '8589934592'}"
            chunk_limit_size "#{ENV['BUFFER_SIZE_LIMIT'] || '8m'}"
            overflow_action block
            disable_chunk_backup true
          </buffer>
        </match>
      </label>
      ....
      Show
      Deploy RHOL $ oc get csv -n openshift-logging NAME                           DISPLAY                            VERSION   REPLACES   PHASE cluster-logging.5.4.3          Red Hat OpenShift Logging          5.4.3                Succeeded elasticsearch- operator .5.4.3   OpenShift Elasticsearch Operator   5.4.3                Succeeded Create a secret containing `ca-bundle.crt`, `tls.crt` and `tls.key` for being used later in the clusterLogForwarder for sending to syslog: $ oc get secret syslog -o yaml  apiVersion: v1 data:   ca-bundle.crt: xxxxx   tls.crt: xxxxx   tls.key: xxxxx kind: Secret metadata:   creationTimestamp: "2022-07-20T15:01:27Z"   name: syslog   namespace: openshift-logging   resourceVersion: "7350054"   uid: 5a4f01b3-e6f8-4f55-a284-9976217810a6 type: Opaque Configure clusterLogForwarder for sending logs to rsyslog using tls   $ cat clusterlogforwarder.yaml apiVersion: logging.openshift.io/v1 kind: ClusterLogForwarder metadata:   labels:     app.kubernetes.io/instance: logging   name: instance   namespace: openshift-logging spec:   outputs:     - name: syslog       secret:         name: syslog       syslog:         addLogSource: true         facility: syslog         rfc: RFC3164         severity: informational       type: syslog       url: 'tls: //localhost:514'   pipelines:     - inputRefs:         - application       name: application-logs       outputRefs:         - syslog $ oc create -f clusterlogforwarder.yaml Check the fluentd configuration generated that it only contains the `ca-bundle.crt` and not the `tls.key` and `tls.crt`: $ oc get cm collector -o json |jq '.data. "fluent.conf" ' -r ... <match journal.** system. var .log**>     @type remote_syslog     @id syslog_journal     host localhost     port 514     rfc rfc3164     facility syslog     severity informational     program ${$.systemd.u.SYSLOG_IDENTIFIER}     protocol tcp     packet_size 4096     hostname "#{ENV[ 'NODE_NAME' ]}"     tls true     ca_file '/ var /run/ocp-collector/secrets/syslog/ca-bundle.crt'     timeout 60     timeout_exception true     keep_alive true     keep_alive_idle 75     keep_alive_cnt 9     keep_alive_intvl 7200     <format>       @type json     </format>     <buffer $.systemd.u.SYSLOG_IDENTIFIER>       @type file       path '/ var /lib/fluentd/syslog_journal'       flush_mode interval       flush_interval 1s       flush_thread_count 2       retry_type exponential_backoff       retry_wait 1s       retry_max_interval 60s       retry_timeout 60m       queued_chunks_limit_size "#{ENV[ 'BUFFER_QUEUE_LIMIT' ] || '32' }"       total_limit_size "#{ENV[ 'TOTAL_LIMIT_SIZE_PER_BUFFER' ] || '8589934592' }"       chunk_limit_size "#{ENV[ 'BUFFER_SIZE_LIMIT' ] || '8m' }"       overflow_action block       disable_chunk_backup true     </buffer>   </match>  <match **>     @type remote_syslog     @id syslog     host localhost     port 514     rfc rfc3164     facility syslog     severity informational     protocol tcp     packet_size 4096     hostname "#{ENV[ 'NODE_NAME' ]}"     tls true     ca_file '/ var /run/ocp-collector/secrets/syslog/ca-bundle.crt'     timeout 60     timeout_exception true     keep_alive true     keep_alive_idle 75     keep_alive_cnt 9     keep_alive_intvl 7200     <format>       @type json     </format>     <buffer>       @type file       path '/ var /lib/fluentd/syslog'       flush_mode interval       flush_interval 1s       flush_thread_count 2       retry_type exponential_backoff       retry_wait 1s       retry_max_interval 60s       retry_timeout 60m       queued_chunks_limit_size "#{ENV[ 'BUFFER_QUEUE_LIMIT' ] || '32' }"       total_limit_size "#{ENV[ 'TOTAL_LIMIT_SIZE_PER_BUFFER' ] || '8589934592' }"       chunk_limit_size "#{ENV[ 'BUFFER_SIZE_LIMIT' ] || '8m' }"       overflow_action block       disable_chunk_backup true     </buffer>   </match> </label> ....
    • Log Collection - Sprint 224, Log Collection - Sprint 225, Log Collection - Sprint 226

      When defined for forwarding logs using the syslog protocol with tls and the secret contains the `ca-bundle.crt`, `tls.key` and `tls.crt`, the real configuration generated for fluentd defines only `ca-bundle.crt` omitting `tls.key` and `tls.cert`.

            syedriko_sub@redhat.com Sergey Yedrikov
            rhn-support-ocasalsa Oscar Casal Sanchez
            Qiaoling Tang Qiaoling Tang
            Votes:
            2 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: