Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-4086

Cluster-logging collector daemonset in CrashLoop when upgraded from 5.6 to 5.7

    XMLWordPrintable

Details

    • False
    • None
    • False
    • NEW
    • NEW
    • Hide
      Before this change the API server would default the new collector type field to "vector" even when the custom resource still wanted to use the old field. This change removes the default for the new collector type field in order to restore the previous behavior
      Show
      Before this change the API server would default the new collector type field to "vector" even when the custom resource still wanted to use the old field. This change removes the default for the new collector type field in order to restore the previous behavior
    • Bug Fix
    • Log Collection - Sprint 236

    Description

      On a cluster partially managed by app-sre via app-interface, the RHOBS team had deployed the cluster logging operator and set up this configuration for the cluster logging resource: https://gitlab.cee.redhat.com/service/app-interface/-/blob/master/resources/setup/cluster-logging/clusterlogging.yaml.

      The cluster log forwarder was configured with the highlighted lines from the following file: https://gitlab.cee.redhat.com/service/app-interface/-/blob/master/resources/rhobs/logs/clusterlogforwarder.yaml.j2#L8-26

      Upon the automatic upgrade of the cluster-logging operator from 5.6 to 5.7, it was noticed that all the pods in the logging collector daemonset were chrashlooping. 

      Here are some details about the crashlooping pods:

      • They were running Vector, surprisingly, even though we had configured fluentd as collector.
      • The following errors appeared in the events:

        • pipeline_0_:
      • lastTransitionTime: '2023-05-10T16:03:34Z'
        message: pipeline must have a name
        reason: Invalid
        status: 'False'
        type: Ready

       

      Attachments

        Activity

          People

            jcantril@redhat.com Jeffrey Cantrill
            rh-ee-doolivei Douglas Camata
            Ishwar Kanse Ishwar Kanse
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: