Details
-
Bug
-
Resolution: Done
-
Normal
-
Logging 5.7.z, Logging 5.7.2
-
None
-
False
-
None
-
False
-
NEW
-
VERIFIED
-
Before this update, the Loki Operator crashed caused when a LokiStack custom resources defined with tenant limits only. With this update, the Loki Operator can process LokiStack custom resources without global limits resolves the issue.
-
Bug Fix
-
Log Storage - Sprint 237
Description
Description
Loki Controller manager crashes when LokiStack CR is created with limits per tenant.
This issue is not seen on 5.7.1
Error:
{"_ts":"2023-06-02T19:47:15.321825092Z","_level":"0","_component":"loki-operator","_message":"Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference","LokiStack":{"name":"lokistack-dev","namespace":"openshift-logging"},"controller":"lokistack","controllerGroup":"loki.grafana.com","controllerKind":"LokiStack","name":"lokistack-dev","namespace":"openshift-logging","reconcileID":"c9979f30-0367-41ff-b3f6-ba753c507257"}
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x1b3971c]
goroutine 742 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
/remote-source/loki-operator/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.5/pkg/internal/controller/controller.go:119 +0x1fa
panic({0x1d2ff00, 0x344f050})
/usr/lib/golang/src/runtime/panic.go:884 +0x213
.....
CSV: loki-operator.v5.7.2
Steps to reproduce:
1) Deploy LO v5.7.2
2) Create LokiStack with per tenant limits.
LokiStack CR:
spec:
limits:
tenants:
application:
ingestion:
ingestionRate: 15
managementState: Managed
size: 1x.extra-small
storage:
schemas:
- effectiveDate: '2022-06-01'
version: v12
secret:
name: test
type: s3
storageClassName: standard-csi
tenants:
mode: openshift-logging
$ oc get pods -n openshift-operators-redhat NAME READY STATUS RESTARTS AGE elasticsearch-operator-54559968db-b2t2k 2/2 Running 0 110m loki-operator-controller-manager-6b44c9bd49-jdxjl 1/2 CrashLoopBackOff 5 (2m20s ago) 20m
How reproducible: Always
Expected Result: LokiStack should be running with no errors and overrides can be seen under runtime-config
Actual Result: Controller Crashes with runtime error
Additional Info:
Container image: registry.redhat.io/openshift-logging/loki-rhel8-operator@sha256:d1a921029b3079f68d052504742d774b287d203a81d5eafafd32185b4f823fa1
Attachments
Issue Links
- clones
-
-
- Closed
-
- links to
- mentioned on
