Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-4564

CLO pod is in CrashLoop when CLF is created without CL instance

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • NEW
    • VERIFIED
    • Release Note Not Required
    • Log Collection - Sprint 242

      Description of problem:
      Cluster Logging operator pod goes into CrashLoop when CLF is created and ClusterLogging instance is not present.
      CL instance needs to be created for the pod to be in running state again.

      $ oc get csv
      NAME                            DISPLAY                            VERSION   REPLACES   PHASE
      cluster-logging.v5.8.0          Red Hat OpenShift Logging          5.8.0                Installing
      elasticsearch-operator.v5.8.0   OpenShift Elasticsearch Operator   5.8.0                Succeeded
      $ oc get pods -w
      NAME                                        READY   STATUS             RESTARTS      AGE
      cluster-logging-operator-6f78974bf8-7dl9t   0/1     CrashLoopBackOff   4 (40s ago)   3m50s
      $ oc logs -f cluster-logging-operator-6f78974bf8-7dl9t
      {"_ts":"2023-09-28T13:12:01.199646457Z","_level":"0","_component":"cluster-logging-operator","_message":"starting up...","go_arch":"amd64","go_os":"linux","go_version":"go1.20.5","operator_version":"5.8.0"} {"_ts":"2023-09-28T13:12:01.220152111Z","_level":"0","_component":"cluster-logging-operator","_message":"migrating resources provided by the manifest"} {"_ts":"2023-09-28T13:12:01.223873932Z","_level":"0","_component":"cluster-logging-operator","_message":"Registering Components."} {"_ts":"2023-09-28T13:12:01.230364438Z","_level":"0","_component":"cluster-logging-operator","_message":"Starting the Cmd."}
      panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
      [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x165e7bb]
      goroutine 454 [running]:
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
        /remote-source/cluster-logging-operator/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:119 +0x1fa
      panic({0x1832ce0, 0x2938b20})
        /usr/lib/golang/src/runtime/panic.go:884 +0x213
      github.com/openshift/cluster-logging-operator/internal/k8s/loader.FetchClusterLogForwarder({, _}, {, }, {, _}, _, _)
        /remote-source/cluster-logging-operator/app/internal/k8s/loader/load.go:66 +0x53b
      github.com/openshift/cluster-logging-operator/controllers/forwarding.(*ReconcileForwarder).Reconcile(0xc000512480, {0x30?, 0xc000680000?}, {0xc005aa3e30?, 0x0?}, {0xc00303b108?, 0x4142c7?})
        /remote-source/cluster-logging-operator/app/controllers/forwarding/forwarding_controller.go:86 +0x3c5
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x1d059e0?, {0x1d059e0?, 0xc00174ce10?}, {0xc005aa3e30?, 0x17b5d60?}, {0xc00303b108?, 0x0?})
        /remote-source/cluster-logging-operator/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:122 +0xc8
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000134280, {0x1d05938, 0xc00010ee60}, {0x18b9620?, 0xc0002c1d60?})
        /remote-source/cluster-logging-operator/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:323 +0x35f
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000134280, {0x1d05938, 0xc00010ee60})
        /remote-source/cluster-logging-operator/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:274 +0x1d9
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
        /remote-source/cluster-logging-operator/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:235 +0x85
      created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
        /remote-source/cluster-logging-operator/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:231 +0x587
      

      Version: Logging 5.8, OCP 4.14 (Used in test)

      How reproducible: Always
      Steps to Reproduce:
      1) Deploy CLO and LO/EO operators
      2) Create CLF

      apiVersion: logging.openshift.io/v1
      kind: ClusterLogForwarder
      metadata:
        name: instance
        namespace: openshift-logging
      spec:
        pipelines:
          - name: all-logs-to-default
            inputRefs:
            - infrastructure
            - application
            - audit
            outputRefs:
            - default

      Actual results: CLF should be created without error under CLO pod

      Expected results: CLO pod in Crashloop status

      Image: 
      registry.redhat.io/openshift-logging/cluster-logging-rhel9-operator@sha256:0388a091331e3b5edf16789aa92fca3f518432d7975226ce1bf5fe0018c85eeb

              jcantril@redhat.com Jeffrey Cantrill
              rhn-support-kbharti Kabir Bharti
              Kabir Bharti Kabir Bharti
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: