Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-3049

[release-5.5] Resources associated with collector / fluentd keep on getting recreated

    XMLWordPrintable

Details

    • False
    • None
    • False
    • NEW
    • VERIFIED
    • Before this update, the operator could enter a loop of removing and recreating the collector daemonset while the Elasticsearch or Kibana deployments changed their status. With this update, a fix in the status handling of the operator resolves the issue.
    • Hide

      Steps to reproduce:

      • Install Red Hat OpenShift Logging and OpenShift Elasticsearch operators on stable-5.4 channel.
      • Upgrade the operators (starting with elasticsearch and then OpenShift Logging) to stable-5.5 channel.
      Show
      Steps to reproduce: Install Red Hat OpenShift Logging and OpenShift Elasticsearch operators on stable-5.4 channel. Upgrade the operators (starting with elasticsearch and then OpenShift Logging) to stable-5.5 channel.
    • Log Collection - Sprint 226

    Description

      After upgrading Red Hat OpenShift Logging from stable-5.4 to stable-5.5, the resources associated with log collector keeps on getting recreated.

      Below are the resources which are getting recreated:
      1. Collector secrets.
      2. Collector pods
      3. Collector daemonset

      State of pods during this issue:
      Terminating -> Pending -> Running -> Terminating

      Below logs are observed in cluster-logging-operator pod:
      ~~~
      {"_ts":"2022-09-08T14:06:43.247384021Z","_level":"0","_component":"cluster-logging-operator","_message":"clusterRequest.reconcileCollectorDaemonset","_error":{"msg":"daemonsets.apps \"collector\" not found"}}
      {"_ts":"2022-09-08T14:06:43.247455916Z","_level":"0","_component":"cluster-logging-operator","_message":"Unable to reconcile collection for \"instance\": daemonsets.apps \"collector\" not found","_error":{"msg":"daemonsets.apps \"collector\" not found"}}
      {"_ts":"2022-09-08T14:06:43.247501321Z","_level":"0","_component":"cluster-logging-operator","_message":"clusterlogforwarder-controller returning, error","_error":{"msg":"Unable to reconcile collection for \"instance\": daemonsets.apps \"collector\" not found"}}
      {"_ts":"2022-09-08T14:08:34.585998336Z","_level":"0","_component":"cluster-logging-operator","_message":"Could not find Secret","Name":"logcollector-token","_error":{"msg":"Secret \"logcollector-token\" not found"}}
      ~~~

      Attachments

        1. events-logging.txt
          8.14 MB
        2. logs-1.txt
          8 kB

        Activity

          People

            rojacob@redhat.com Robert Jacob
            rhn-support-dgautam Dhruv Gautam
            Qiaoling Tang Qiaoling Tang
            Votes:
            9 Vote for this issue
            Watchers:
            24 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: