-
Bug
-
Resolution: Done
-
Blocker
-
Logging 5.5.6
-
False
-
None
-
False
-
NEW
-
VERIFIED
-
-
-
-
Rox Sprint 74C - Global, Log Collection - Sprint 229
-
Important
Description of problem:
v5.5.6-8 cluster-logging-operator pod ends up in CrashLoopBackOff status.
[kni@registry.kni-qe-0 ~]$ oc -n openshift-logging get pods
NAME READY STATUS RESTARTS AGE
cluster-logging-operator-7cbd77bf-r5ksv 0/1 CrashLoopBackOff 12 (3m28s ago) 69m
collector-tnmqg 2/2 Running 0 68m
[kni@registry.kni-qe-0 ~]$ oc -n openshift-logging logs cluster-logging-operator-7cbd77bf-r5ksv
{"_ts":"2022-12-14T11:11:37.225541809Z","_level":"0","_component":"cluster-logging-operator","_message":"starting up...","go_arch":"amd64","go_os":"linux","go_version":"go1.17.12","operator_version":"5.5"} I1214 11:11:38.276655 1 request.go:665] Waited for 1.039760579s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/security.internal.openshift.io/v1?timeout=32s {"_ts":"2022-12-14T11:11:39.932221196Z","_level":"0","_component":"cluster-logging-operator","_message":"migrating resources provided by the manifest"} {"_ts":"2022-12-14T11:11:39.934778181Z","_level":"0","_component":"cluster-logging-operator","_message":"Registering Components."} {"_ts":"2022-12-14T11:11:39.935175808Z","_level":"0","_component":"cluster-logging-operator","_message":"Starting the Cmd."} I1214 11:11:55.186275 1 request.go:665] Waited for 1.046884604s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/image.openshift.io/v1?timeout=32s I1214 11:12:05.187029 1 request.go:665] Waited for 1.045809582s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/machine.openshift.io/v1beta1?timeout=32s I1214 11:12:15.236201 1 request.go:665] Waited for 1.097533537s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/authorization.k8s.io/v1?timeout=32s I1214 11:12:25.236923 1 request.go:665] Waited for 1.096393882s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/project.openshift.io/v1?timeout=32s I1214 11:12:35.286178 1 request.go:665] Waited for 1.146796421s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/autoscaling/v2beta2?timeout=32s I1214 11:12:45.336396 1 request.go:665] Waited for 1.197389464s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/local.storage.openshift.io/v1?timeout=32s I1214 11:12:55.336482 1 request.go:665] Waited for 1.198072466s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/coordination.k8s.io/v1?timeout=32s I1214 11:13:05.387352 1 request.go:665] Waited for 1.224493028s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/local.storage.openshift.io/v1?timeout=32s {"_ts":"2022-12-14T11:13:11.017893653Z","_level":"0","_component":"cluster-logging-operator","_message":"clusterlogforwarder-controller error updating status","_error":{"msg":"Operation cannot be fulfilled on clusterlogforwarders.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again"}} I1214 11:13:15.437612 1 request.go:665] Waited for 1.22207017s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/operators.coreos.com/v1?timeout=32s I1214 11:13:25.486911 1 request.go:665] Waited for 1.348064753s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/sriovnetwork.openshift.io/v1?timeout=32s I1214 11:13:35.536707 1 request.go:665] Waited for 1.397478125s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/performance.openshift.io/v1alpha1?timeout=32s {"_ts":"2022-12-14T11:13:36.889681986Z","_level":"0","_component":"cluster-logging-operator","_message":"clusterlogforwarder-controller error updating status","_error":{"msg":"Operation cannot be fulfilled on clusterlogforwarders.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again"}} I1214 11:13:45.536951 1 request.go:665] Waited for 1.397949876s due to client-side throttling, not priority and fairness, request: GET:https://[fd02::1]:443/apis/admissionregistration.k8s.io/v1?timeout=32s {"_ts":"2022-12-14T11:14:01.024031054Z","_level":"0","_component":"cluster-logging-operator","_message":"Manager exited non-zero","_error":{"msg":"failed to wait for clusterlogging caches to sync: timed out waiting for cache to be synced"}}
Version-Release number of selected component (if applicable):
cluster-logging-operator-metadata-container-v5.5.6-8
registry-proxy.engineering.redhat.com/rh-osbs/openshift-logging-cluster-logging-rhel8-operator:v5.5.6-4
{ ....... "architecture": "x86_64", "build-date": "2022-12-13T15:25:42", "io.openshift.build.commit.id": "ce22bf231b89df7923dc45666cb220a8c29eca1b", "io.openshift.build.commit.url": "https://github.com/openshift/cluster-logging-operator/commit/ce22bf231b89df7923dc45666cb220a8c29eca1b", "name": "openshift-logging/cluster-logging-rhel8-operator", "release": "4", "summary": "Provides the latest release of Red Hat Universal Base Image 8.", "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/openshift-logging/cluster-logging-rhel8-operator/images/v5.5.6-4", "vcs-ref": "f1d8d61ee49159769e555234548ce7a93d24fa46", "vcs-type": "git", "vendor": "Red Hat, Inc.", "version": "v5.5.6" }
4.12.0-rc.4, Single node and IPv6 Single stack cluster
How reproducible:
always
Steps to Reproduce:
- Install and configure cluster-logging on an SNO IPv6 environment with Telco DU profile applied
- Check cluster-logging-operator pod status
Actual results:
Pod gets restarted and ends up in CrashLoopBackOff
Expected results:
No failures.
Additional info:
Attaching output of operator image info and pod describe.pod-info.txt
- clones
-
LOG-3406 Cluster Logging Operator does not start successfully when Elasticsearch resource is not available
- Closed
- is related to
-
LOG-3321 CLO pods restarting freqently and throw client-side throttling, not priority and fairness
- Closed
- relates to
-
LOG-2999 [ClusterLogging] elasticsearchStatus in ClusterLogging instance CR is not updated when Elasticsearch status is changed
- Closed
- links to
- mentioned on