-
Bug
-
Resolution: Done
-
Critical
-
Logging 5.6.24
-
False
-
None
-
False
-
NEW
-
NEW
-
-
-
Log Storage - Sprint 260
-
Critical
Description of problem:
Loki pods are in CrashLoopBackOff when the cluster has FIPS enabled.
oc get pods NAME READY STATUS RESTARTS AGE cluster-logging-operator-df84b4bcf-tqxcr 1/1 Running 0 54m lokistack-sample-compactor-0 0/1 CrashLoopBackOff 7 (95s ago) 13m lokistack-sample-distributor-7597797f68-tznhz 0/1 CrashLoopBackOff 7 (117s ago) 13m lokistack-sample-gateway-854d87d7d7-779lz 2/2 Running 0 13m lokistack-sample-gateway-854d87d7d7-8xrgw 2/2 Running 0 13m lokistack-sample-index-gateway-0 0/1 CrashLoopBackOff 7 (2m9s ago) 13m lokistack-sample-ingester-0 0/1 CrashLoopBackOff 7 (106s ago) 13m lokistack-sample-querier-768d77d494-6whm9 0/1 CrashLoopBackOff 7 (116s ago) 13m lokistack-sample-query-frontend-59cdcbcdd6-xvz8x 0/1 CrashLoopBackOff 7 (2m20s ago) 13m lokistack-sample-ruler-0 0/1 CrashLoopBackOff 7 (118s ago) 13m oc logs lokistack-sample-compactor-0panic: opensslcrypto: can't initialize OpenSSL : openssl: can't retrieve OpenSSL version goroutine 1 [running]:crypto/internal/backend.init.0() /usr/lib/golang/src/crypto/internal/backend/openssl.go:50 +0x244
Version-Release number of selected component (if applicable):
loki-operator.v5.6.24
Cluster version: 4.13.0-0.nightly-arm64-2024-09-26-214441
How reproducible:
Always
Steps to Reproduce:
1. deploy lokistack
oc get lokistack lokistack-sample -o yaml |yq '.spec' managementState: Managed rules: enabled: true namespaceSelector: matchLabels: openshift.io/cluster-monitoring: "true" selector: matchLabels: openshift.io/cluster-monitoring: "true" size: 1x.extra-small storage: schemas: - effectiveDate: "2020-10-11" version: v11 secret: name: s3-secret type: s3 storageClassName: gp3-csi tenants: mode: openshift-logging
2. check loki pods status
Actual results:
Expected results:
Loki pods should be ready
Additional info:
- links to
-
RHBA-2024:138714
Logging for Red Hat OpenShift - 5.8.13
- mentioned on
[LOG-6127] [Logging 5.6.24]Loki pods are in CrashLoopBackOff
Since the problem described in this issue should be resolved in a recent advisory, it has been closed.
For information on the advisory (Moderate: Logging for Red Hat OpenShift - 5.8.13), and where to find the updated files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHSA-2024:7237
The issue is fixed in loki-operator.v5.8.13 and loki-operator.v5.9.7.
GitLab CEE Bot
added a comment - Periklis Tsirakidis mentioned this issue in merge request !4535 of openshift-logging / Log Storage Midstream on branch upstream-loki-v3.1.1-5.9-on-go1.21:
LOG-6127: Move Loki builds back to go1.21 (5.9.7)
GitLab CEE Bot
added a comment - Periklis Tsirakidis mentioned this issue in merge request !4535 of openshift-logging / Log Storage Midstream on branch upstream-loki-v3.1.1-5.9-on-go1.21 : LOG-6127 : Move Loki builds back to go1.21 (5.9.7) Ok the cause of this one is the recent update of Loki 3.1.1 and built with go1.22 (See LOG-5950). Both are unreleased yet, so no impact on customers. Unfortunately RHEL 9.z is still pending a couple of fixes for go1.22 and underlying components like runc which makes it currently impossible to use go1.22 on any OCP version (See OCPBUGS-36541 and RHEL-46380).
For now I am going to keep Loki 3.1.1 on the z-streams but downgrade go to 1.21.
GitLab CEE Bot
added a comment - Periklis Tsirakidis mentioned this issue in merge request !4531 of openshift-logging / Log Storage Midstream on branch upstream-loki-v3.1.1-5.8-on-go1.21:
LOG-6127: Move Loki builds back to go1.21 (5.8.13)
GitLab CEE Bot
added a comment - Periklis Tsirakidis mentioned this issue in merge request !4531 of openshift-logging / Log Storage Midstream on branch upstream-loki-v3.1.1-5.8-on-go1.21 : LOG-6127 : Move Loki builds back to go1.21 (5.8.13) GitLab CEE Bot
added a comment - Gabriel Bernal mentioned this issue in merge request !4352 of openshift-logging / Log Collection Midstream on branch fix-node-version-console-plugin:
LOG-6127: upgrade node version for logging view plugin
GitLab CEE Bot
added a comment - Gabriel Bernal mentioned this issue in merge request !4352 of openshift-logging / Log Collection Midstream on branch fix-node-version-console-plugin : LOG-6127 : upgrade node version for logging view plugin This issue requires Release Notes Text. Please modify the Release Note Text or set the Release Note Type to "Release Note Not Required"
GitLab CEE Bot
added a comment - Gabriel Bernal mentioned this issue in merge request !4354 of openshift-logging / Log Collection Midstream on branch fix-node-version-plugin-5.8:
LOG-6127: fix node version and add FIPS compliance for console plugin
GitLab CEE Bot
added a comment - Gabriel Bernal mentioned this issue in merge request !4354 of openshift-logging / Log Collection Midstream on branch fix-node-version-plugin-5.8 : LOG-6127 : fix node version and add FIPS compliance for console plugin
Periklis Tsirakidis mentioned this issue in merge request !4566 of openshift-logging / Log Storage Midstream on branch upstream-loki-v3.1.1-5.6-on-go1.21: