Details
-
Bug
-
Resolution: Done
-
Blocker
-
Logging 5.5.4
-
False
-
None
-
False
-
NEW
-
VERIFIED
-
Log Storage - Sprint 227
Description
Description of problem:
Loki pod is going to 'CrashLoopBackOff' state, reporting "tls-secret" : secret "lokistack-sample-gateway-http" not found even though the secret is present.
Version-Release number of selected component (if applicable):
Logging 5.5.4
How reproducible:
Always
Steps to Reproduce:
- Install CLO and Loki
- Create ClusterLogging instance
- Deploy lokistack components under openshift-logging
Actual results:
$ oc get pods
NAME READY STATUS RESTARTS AGE
cluster-logging-operator-6f488b8f7c-gl7zr 1/1 Running 0 10m
collector-2l24n 2/2 Running 0 3m46s
collector-cl8mj 2/2 Running 0 3m41s
collector-dvt84 2/2 Running 0 3m51s
collector-fvkzw 2/2 Running 0 3m50s
collector-kqcxl 2/2 Running 0 3m44s
collector-sswcr 2/2 Running 0 3m50s
logging-view-plugin-67d767869f-zxd4b 1/1 Running 0 4m21s
lokistack-sample-compactor-0 1/1 Running 0 4m36s
lokistack-sample-distributor-664bfc8648-bld7m 1/1 Running 0 4m36s
lokistack-sample-gateway-859587754-q5xd5 1/2 CrashLoopBackOff 5 (77s ago) 4m35s
lokistack-sample-index-gateway-0 1/1 Running 0 4m35s
lokistack-sample-ingester-0 1/1 Running 0 4m36s
lokistack-sample-querier-6d8864b4c7-qgrd5 1/1 Running 0 4m36s
lokistack-sample-query-frontend-856fdb65d8-fv8l6 1/1 Running 0 4m36s
$ oc describe pod lokistack-sample-gateway-859587754-q5xd5 ... ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 2m47s default-scheduler Successfully assigned openshift-logging/lokistack-sample-gateway-859587754-q5xd5 to ip-10-0-192-19.us-east-2.compute.internal by ip-10-0-161-253 Warning FailedMount 2m47s kubelet MountVolume.SetUp failed for volume "lokistack-sample-ca-bundle" : configmap "lokistack-sample-ca-bundle" not found Warning FailedMount 2m45s (x3 over 2m47s) kubelet MountVolume.SetUp failed for volume "tls-secret" : secret "lokistack-sample-gateway-http" not found Normal AddedInterface 2m42s multus Add eth0 [10.128.2.21/23] from openshift-sdn Normal Pulling 2m42s kubelet Pulling image "registry.redhat.io/openshift-logging/lokistack-gateway-rhel8@sha256:c27ac984ce6f16100300b48919b1c9d14d9cb7531352bf27cc7052b9db2bbaff" Normal Pulled 2m37s kubelet Successfully pulled image "registry.redhat.io/openshift-logging/lokistack-gateway-rhel8@sha256:c27ac984ce6f16100300b48919b1c9d14d9cb7531352bf27cc7052b9db2bbaff" in 4.988485142s Normal Pulling 2m37s kubelet Pulling image "registry.redhat.io/openshift-logging/opa-openshift-rhel8@sha256:88ad9c400d9dc49b070f9b7c44641fab456eed718b563a06cd77d68faf3835d4" Normal Pulled 2m32s kubelet Successfully pulled image "registry.redhat.io/openshift-logging/opa-openshift-rhel8@sha256:88ad9c400d9dc49b070f9b7c44641fab456eed718b563a06cd77d68faf3835d4" in 4.884231335s Normal Created 2m32s kubelet Created container opa Normal Started 2m32s kubelet Started container opa Warning BackOff 2m28s (x3 over 2m30s) kubelet Back-off restarting failed container Normal Pulled 2m15s (x2 over 2m31s) kubelet Container image "registry.redhat.io/openshift-logging/lokistack-gateway-rhel8@sha256:c27ac984ce6f16100300b48919b1c9d14d9cb7531352bf27cc7052b9db2bbaff" already present on machine Normal Created 2m14s (x3 over 2m37s) kubelet Created container gateway Normal Started 2m14s (x3 over 2m37s) kubelet Started container gateway Warning Unhealthy 2m14s kubelet Readiness probe failed: Get "https://10.128.2.21:8081/ready": dial tcp 10.128.2.21:8081: connect: connection refused Warning ProbeError 2m13s (x2 over 2m14s) kubelet Readiness probe error: Get "https://10.128.2.21:8081/ready": dial tcp 10.128.2.21:8081: connect: connection refused body: Warning ProbeError 2m13s kubelet Liveness probe error: Get "https://10.128.2.21:8081/live": dial tcp 10.128.2.21:8081: connect: connection refused body:
$ oc get secret |grep lokistack-sample-gateway-http lokistack-sample-gateway-http kubernetes.io/tls 2 7m22s
Expected results:
All loki pods are up and running
Attachments
Issue Links
- links to