Details
-
Bug
-
Resolution: Done
-
Normal
-
Logging 5.4.z, Logging 5.4.0
-
None
-
False
-
None
-
False
-
NEW
-
OBSDA-7 - Adopting Loki as an alternative to Elasticsearch to support more lightweight, easier to manage/operate storage scenarios
-
NEW
-
Logging (LogExp) - Sprint 216, Logging (LogExp) - Sprint 217
Description
Reconcile Error on controller pod when LokiStack size is changed.
Logs: Controller logs
{"_ts":"2022-03-22T03:26:54.721551012Z","_level":"0","_component":"loki-operator_controller-runtime_manager_controller_lokistack","_message":"Reconciler error","_error":\{"msg":"failed to configure lokistack resources","name":{"Namespace":"openshift-logging","Name":"lokistack-dev"}},"name":"lokistack-dev","namespace":"openshift-logging"}.
Steps to reproduce:
1) Deploy Loki Operator from grafana/loki repo using make olm-deploy.
2) Create Loki secret and provision Loki Stack with 1x-extra-small t-shirt size.
spec: managementState: Managed replicationFactor: 1 size: 1x.extra-small storage: secret: name: test type: s3 storageClassName: gp2 tenants: mode: openshift-logging
3) Edit the LokiStack CR and update t-shirt size to 1x.small and replication factor to 2 for HA setup.
$ oc edit LokiStack my-loki-n openshift-logging
spec: managementState: Managed replicationFactor: 2 size: 1x.small storage: secret: name: test type: s3 storageClassName: gp2 tenants: mode: openshift-logging
Expected Results:
a) LokiStack components are provisioned with updated t-shirt size requirements.
Actual Result:
a) LokiStack components are not reconciled correctly.
b) In above case, we expect atleast two index-gateways and ingesters but only one are running after edit. If LokiStack is deleted and recreated with t-shirt size 1x.small, the components are created correctly.
c) PVC requirements are not updated for change of LokiStack Size.
[kbharti@cube hack]$ oc get pods NAME READY STATUS RESTARTS AGE lokistack-dev-compactor-0 1/1 Running 0 29m lokistack-dev-distributor-6f4d7d9bf8-5l97p 1/1 Running 0 26m lokistack-dev-distributor-6f4d7d9bf8-7rszn 1/1 Running 0 27m lokistack-dev-gateway-fc4ff4fbc-g5jmx 2/2 Running 0 27m lokistack-dev-index-gateway-0 1/1 Running 0 29m lokistack-dev-ingester-0 1/1 Running 0 29m lokistack-dev-querier-6b668f995-jhsmc 1/1 Running 0 26m lokistack-dev-querier-6b668f995-s4bfj 1/1 Running 0 27m lokistack-dev-query-frontend-5797d84c-8t9r8 1/1 Running 0 26m lokistack-dev-query-frontend-5797d84c-dq57g 1/1 Running 0 27m
Attachments
Issue Links
- links to