Uploaded image for project: 'Cluster Observability Operator'
  1. Cluster Observability Operator
  2. COO-736

observability-operator pod CrashLoopBackOff for Readiness probe error

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Blocker Blocker
    • 1.1.0
    • None
    • operator
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • None

      Install coo with the following FBC image on OCP 4.18 azure cluster
      coo-fbc-v4-18 lastPromotedImage: quay.io/redhat-user-workloads/cluster-observabilit-tenant/cluster-observability-operator/coo-fbc-v4-18@sha256:b4fab8f832bf2a62e212df1d1b3e812f87e6ee0b2394cc4070c8a3a0bc99601c

      % oc get pod                                               
      NAME                                                         READY   STATUS             RESTARTS      AGE
      obo-prometheus-operator-75494c7f99-kg5dv                     1/1     Running            0             69s
      obo-prometheus-operator-admission-webhook-7f5c74cd87-2k7lx   1/1     Running            0             69s
      obo-prometheus-operator-admission-webhook-7f5c74cd87-jw69n   1/1     Running            0             69s
      observability-operator-7f7548b84b-q7xqz                      0/1     CrashLoopBackOff   2 (33s ago)   69s
      perses-operator-7d8cdd7658-qg989                             1/1     Running            0             68s
      
      % oc describe pod observability-operator-7f7548b84b-q7xqz
      Name:             observability-operator-7f7548b84b-q7xqz
      Namespace:        openshift-cluster-observability-operator
      Priority:         0
      Service Account:  observability-operator-sa
      Node:             lhy-418-312-zn2sc-worker-eastus2-47nqt/10.0.128.4
      Start Time:       Wed, 12 Mar 2025 11:01:50 +0800
      Labels:           app.kubernetes.io/component=operator
                        app.kubernetes.io/name=observability-operator
                        app.kubernetes.io/version=0.0.1
                        pod-template-hash=7f7548b84b 
      ......
      Events:
        Type     Reason          Age   From               Message
        ----     ------          ----  ----               -------
        Normal   Scheduled       86s   default-scheduler  Successfully assigned openshift-cluster-observability-operator/observability-operator-7f7548b84b-q7xqz to lhy-418-312-zn2sc-worker-eastus2-47nqt
        Normal   AddedInterface  86s   multus             Add eth0 [10.131.0.27/23] from ovn-kubernetes
        Normal   Pulled          78s   kubelet            Successfully pulled image "quay.io/redhat-user-workloads/cluster-observabilit-tenant/cluster-observability-operator/cluster-observability-rhel8-operator@sha256:5adfa77852067d51faabc2d627b515927a665a26e8bfcd22933cf178508280cd" in 7.937s (7.937s including waiting). Image size: 379509731 bytes.
        Warning  Unhealthy       76s   kubelet            Liveness probe failed: Get "http://10.131.0.27:8081/healthz": dial tcp 10.131.0.27:8081: connect: connection refused
        Warning  ProbeError      76s   kubelet            Liveness probe error: Get "http://10.131.0.27:8081/healthz": dial tcp 10.131.0.27:8081: connect: connection refused
      body:
        Normal   Pulled      75s                kubelet  Successfully pulled image "quay.io/redhat-user-workloads/cluster-observabilit-tenant/cluster-observability-operator/cluster-observability-rhel8-operator@sha256:5adfa77852067d51faabc2d627b515927a665a26e8bfcd22933cf178508280cd" in 740ms (740ms including waiting). Image size: 379509731 bytes.
        Warning  BackOff     66s (x3 over 73s)  kubelet  Back-off restarting failed container operator in pod observability-operator-7f7548b84b-q7xqz_openshift-cluster-observability-operator(b4ed00b6-fbcb-4d74-a5e5-5efc65c56e04)
        Normal   Pulling     54s (x3 over 86s)  kubelet  Pulling image "quay.io/redhat-user-workloads/cluster-observabilit-tenant/cluster-observability-operator/cluster-observability-rhel8-operator@sha256:5adfa77852067d51faabc2d627b515927a665a26e8bfcd22933cf178508280cd"
        Normal   Pulled      53s                kubelet  Successfully pulled image "quay.io/redhat-user-workloads/cluster-observabilit-tenant/cluster-observability-operator/cluster-observability-rhel8-operator@sha256:5adfa77852067d51faabc2d627b515927a665a26e8bfcd22933cf178508280cd" in 953ms (953ms including waiting). Image size: 379509731 bytes.
        Normal   Created     52s (x3 over 77s)  kubelet  Created container: operator
        Normal   Started     52s (x3 over 77s)  kubelet  Started container operator
        Warning  ProbeError  52s (x4 over 77s)  kubelet  Readiness probe error: Get "http://10.131.0.27:8081/healthz": dial tcp 10.131.0.27:8081: connect: connection refused
      body:
        Warning  Unhealthy  52s (x4 over 77s)  kubelet  Readiness probe failed: Get "http://10.131.0.27:8081/healthz": dial tcp 10.131.0.27:8081: connect: connection refused

       

       

              jfajersk@redhat.com Jan Fajerski
              hongyli@redhat.com Hongyan Li
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: