Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-6585

Collectors are in error when unused outputs are defined on the CLF

XMLWordPrintable

    • False
    • None
    • False
    • NEW
    • NEW
    • Hide
      Before this change the operator could deploy the collector with output configurations that are not referenced by any inputs. This update resolves that by adding a validation to fail the ClusterLogForwarder in this scenario so the operator will no longer deploy the collector.
      Show
      Before this change the operator could deploy the collector with output configurations that are not referenced by any inputs. This update resolves that by adding a validation to fail the ClusterLogForwarder in this scenario so the operator will no longer deploy the collector.
    • Bug Fix
    • Log Collection - Sprint 266
    • Moderate
    • QE Confirmed, Customer Reported

      Description of problem:

      When unused outputs are provisioned under CLF, the collectors are in error/CrashLoop status with below issue

      Creating the directory used for persisting Vector state /var/lib/vector/openshift-logging/logging-clf
      Checking for buffer lock files
      /var/lib/vector/openshift-logging/logging-clf /usr/bin
      found lock files: ''
      /usr/bin
      Starting Vector process...
      2025-01-17T21:20:06.914687Z ERROR vector::cli: Configuration error. error=Transform "output_elasticsearch_secure_external_index" has no inputs
      2025-01-17T21:20:06.914707Z ERROR vector::cli: Configuration error. error=Transform "output_ocplogstore_syslogout_parse_encoding" has no inputs

       

      $ oc get csv
      NAME                                   DISPLAY                          VERSION   REPLACES                               PHASE
      cluster-logging.v6.0.3                 Red Hat OpenShift Logging        6.0.3     cluster-logging.v5.9.10                Succeeded
      cluster-observability-operator.0.4.1   Cluster Observability Operator   0.4.1     cluster-observability-operator.0.3.2   Succeeded
      loki-operator.v6.0.3                   Loki Operator                    6.0.3     loki-operator.v5.9.10                  Succeeded
      $ oc get pods -n openshift-logging
      NAME                                            READY   STATUS             RESTARTS      AGE
      cluster-logging-operator-8f4b9646f-xh6pl        1/1     Running            0             23m
      logging-clf-4z6tt                               0/1     CrashLoopBackOff   5 (39s ago)   4m31s
      logging-clf-6mtl8                               0/1     CrashLoopBackOff   5 (38s ago)   4m31s
      logging-clf-76lrw                               0/1     CrashLoopBackOff   5 (43s ago)   4m31s
      logging-clf-l4zvz                               0/1     CrashLoopBackOff   5 (30s ago)   4m30s
      logging-clf-nkzj6                               0/1     CrashLoopBackOff   5 (28s ago)   4m31s
      logging-clf-pbns7                               0/1     CrashLoopBackOff   5 (29s ago)   4m31s
      lokistack-dev-compactor-0                       1/1     Running            0             22m
      lokistack-dev-distributor-57dc89f46b-jgvls      1/1     Running            0             23m
      lokistack-dev-gateway-7844d76b75-2npc7          2/2     Running            0             23m
      lokistack-dev-gateway-7844d76b75-gqjsg          2/2     Running            0             23m
      lokistack-dev-index-gateway-0                   1/1     Running            0             22m
      lokistack-dev-ingester-0                        1/1     Running            0             22m
      lokistack-dev-querier-864b8bdd76-tpwhk          1/1     Running            0             23m
      lokistack-dev-query-frontend-5f6ddd4797-h2b9j   1/1     Running            0             23m
      lokistack-dev-ruler-0                           1/1     Running            0             22m
      

       

      Version-Release number of selected component (if applicable): Logging 6.0.3

      How reproducible: Always

      Steps to Reproduce:
      a) Deploy CLO and LO v6.0
      b) Create CLF to forward logs to LokiStack and another secondary logstore
      c) Only send logs to default LokiStack but to the secondary logStore
      d) Observe collector status

      Actual results:
      Collectors go into crashloop error

      Expected results:
      CLF should be provisioned successfully as a user can define outputs that can be used in the future

      Additional info:
      Elasticsearch and syslog outputs are defined for a future use under the below CLF

      apiVersion: observability.openshift.io/v1
      kind: ClusterLogForwarder
      metadata:
        name: logging-clf
        namespace: openshift-logging
      spec:
        collector:
          tolerations:
            - effect: NoSchedule
              key: node-role.kubernetes.io/infra
              value: reserved
            - effect: NoExecute
              key: node-role.kubernetes.io/infra
              value: reserved
        managementState: Managed
        outputs:
          - lokiStack:
              authentication:
                token:
                  from: serviceAccount
              target:
                namespace: openshift-logging
                namespace: lokistack-dev
            name: logging-loki
            tls:
              ca:
                configMapName: openshift-service-ca.crt
                key: service-ca.crt
            type: lokiStack
          - elasticsearch:
              index: '{.log_type||"default"}-write'
              url: 'https://rhoslog.abc.xyz.ie:9200'
              version: 8
            name: elasticsearch-secure-external
            type: elasticsearch
          - name: ocplogstore-syslogout
            syslog:
              appName: myapp
              facility: user
              rfc: RFC5424
              severity: debug
              url: 'tcp://ocplogstore.abc.xyz.ie:514'
            type: syslog
        pipelines:
          - inputRefs:
              - application
              - infrastructure
            name: logs-to-loki
            outputRefs:
              - logging-loki
        serviceAccount:
          name: my-loki-sa
      status:
        conditions:
          - lastTransitionTime: '2025-01-17T21:03:00Z'
            message: 'permitted to collect log types: [application infrastructure]'
            reason: ClusterRolesExist
            status: 'True'
            type: observability.openshift.io/Authorized
          - lastTransitionTime: '2025-01-17T21:03:00Z'
            message: ''
            reason: ValidationSuccess
            status: 'True'
            type: observability.openshift.io/Valid
          - lastTransitionTime: '2025-01-17T21:03:06Z'
            message: ''
            reason: ReconciliationComplete
            status: 'True'
            type: Ready
        inputConditions:
          - lastTransitionTime: '2025-01-17T21:28:33Z'
            message: input "application" is valid
            reason: ValidationSuccess
            status: 'True'
            type: observability.openshift.io/ValidInput-application
          - lastTransitionTime: '2025-01-17T21:28:33Z'
            message: input "infrastructure" is valid
            reason: ValidationSuccess
            status: 'True'
            type: observability.openshift.io/ValidInput-infrastructure
        outputConditions:
          - lastTransitionTime: '2025-01-17T21:03:00Z'
            message: output "elasticsearch-secure-external" is valid
            reason: ValidationSuccess
            status: 'True'
            type: observability.openshift.io/ValidOutput-elasticsearch-secure-external
          - lastTransitionTime: '2025-01-17T21:03:00Z'
            message: output "ocplogstore-syslogout" is valid
            reason: ValidationSuccess
            status: 'True'
            type: observability.openshift.io/ValidOutput-ocplogstore-syslogout
          - lastTransitionTime: '2025-01-17T21:28:33Z'
            message: output "logging-loki-application" is valid
            reason: ValidationSuccess
            status: 'True'
            type: observability.openshift.io/ValidOutput-logging-loki-application
          - lastTransitionTime: '2025-01-17T21:28:33Z'
            message: output "logging-loki-infrastructure" is valid
            reason: ValidationSuccess
            status: 'True'
            type: observability.openshift.io/ValidOutput-logging-loki-infrastructure
        pipelineConditions:
          - lastTransitionTime: '2025-01-17T21:03:00Z'
            message: pipeline "logs-to-loki" is valid
            reason: ValidationSuccess
            status: 'True'
            type: observability.openshift.io/ValidPipeline-logs-to-loki
          - lastTransitionTime: '2025-01-17T21:28:33Z'
            message: pipeline "logs-to-loki-1" is valid
            reason: ValidationSuccess
            status: 'True'
            type: observability.openshift.io/ValidPipeline-logs-to-loki-1
      
      

       

              jcantril@redhat.com Jeffrey Cantrill
              rhn-support-kbharti Kabir Bharti
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: