Details
-
Bug
-
Resolution: Done
-
Normal
-
Logging 5.6.2
-
False
-
None
-
False
-
NEW
-
NEW
-
Before this update, the `nodeSelector` attribute for the Gateway component of LokiStack did not have any effect. With this update, the `nodeSelector` attribute functions as expected.
-
Log Storage - Sprint 233, Log Storage - Sprint 234
Description
Description of problem:
When applying nodeSelector to gateway the change is not picked-up and applied. The change is done based onnodeSelector for gateway not applied/not working as expected. The change is done according to https://loki-operator.dev/docs/api.md/#loki-grafana-com-v1-LokiTemplateSpec but somehow not working for the gateway component. All the other components it works but gateway fails to apply the change.
{
"apiVersion": "loki.grafana.com/v1",
"kind": "LokiStack",
"metadata": {
"annotations": {
"loki.grafana.com/rulesDiscoveredAt": "2023-02-27T13:37:04Z"
},
"creationTimestamp": "2023-02-27T13:14:19Z",
"generation": 4,
"name": "logging-loki",
"namespace": "openshift-logging",
"resourceVersion": "1547058",
"uid": "338bc979-bcc2-4996-ba67-ae0260c0a8fa"
},
"spec": {
"managementState": "Managed",
"size": "1x.extra-small",
"storage": {
"schemas": [
{
"effectiveDate": "2022-06-01",
"version": "v12"
}
],
"secret": {
"name": "azure-secret",
"type": "azure"
}
},
"storageClassName": "managed-csi",
"template": {
"distributor": {
"nodeSelector": {
"test": "foo"
}
},
"gateway": {
"nodeSelector": {
"test": "foo"
}
}
},
"tenants": {
"mode": "openshift-logging"
}
},
"status": {
"components": {
"compactor": {
"Running": [
"logging-loki-compactor-0"
]
},
"distributor": {
"Running": [
"logging-loki-distributor-7b8dbc758f-nhhgc",
"logging-loki-distributor-65b7cf5c84-wl6c2"
]
},
"gateway": {
"Running": [
"logging-loki-gateway-89d6d5ffc-scvpz",
"logging-loki-gateway-89d6d5ffc-dsk6f"
]
},
"indexGateway": {
"Running": [
"logging-loki-index-gateway-0"
]
},
"ingester": {
"Running": [
"logging-loki-ingester-0"
]
},
"querier": {
"Running": [
"logging-loki-querier-77b5f46675-9xzqx"
]
},
"queryFrontend": {
"Running": [
"logging-loki-query-frontend-6447bd9796-rstms"
]
}
},
"conditions": [
{
"lastTransitionTime": "2023-02-27T14:05:59Z",
"message": "Invalid object storage secret contents: missing secret field",
"reason": "InvalidObjectStorageSecret",
"status": "False",
"type": "Degraded"
},
{
"lastTransitionTime": "2023-02-27T14:05:59Z",
"message": "All components ready",
"reason": "ReadyComponents",
"status": "True",
"type": "Ready"
},
{
"lastTransitionTime": "2023-02-27T14:05:59Z",
"message": "Some LokiStack components pending on dependencies",
"reason": "PendingComponents",
"status": "False",
"type": "Pending"
}
],
"storage": {
"schemas": [
{
"effectiveDate": "2022-06-01",
"version": "v12"
}
]
}
}
}
$ oc get deployment logging-loki-distributor -o json | jq '.spec.template.spec.nodeSelector' { "test": "foo" }
$ oc get deployment logging-loki-gateway -o json | jq '.spec.template.spec.nodeSelector' null
Being able to apply a nodeSelector is crucial to have all components run on specific OpenShift Container Platform 4 - infra nodes (subscription related topic).
The Loki Operator does not reveal details why the nodeSelector is not applied. It shows that the gateway component is updated but is missing details as to why the nodeSelector is not done.
Version-Release number of selected component (if applicable):
loki-operator.v5.6.2
How reproducible:
Always
Steps to Reproduce:
- Install LokiStack and apply a configruation as shown above
Actual results:
$ oc get deployment logging-loki-gateway -o json | jq '.spec.template.spec.nodeSelector' null
Expected results:
$ oc get deployment logging-loki-gateway -o json | jq '.spec.template.spec.nodeSelector' { "test": "foo" }
Additional info:
Interesting the logging-loki-distributor works and is now running on the OpenShift Container Platform 4 - Node with the appopriate label. But the status of the LokiStack resource is showing the old and new pod.
$ oc get lokistack logging-loki -o json | jq '.status.components.distributor' { "Running": [ "logging-loki-distributor-7b8dbc758f-nhhgc", "logging-loki-distributor-65b7cf5c84-wl6c2" ] }
$ oc get pod
NAME READY STATUS RESTARTS AGE
cluster-logging-operator-b8b64cd8f-kxfhs 1/1 Running 0 70m
collector-47cqm 2/2 Running 0 17m
collector-6w5rn 2/2 Running 0 40m
collector-fzc8q 2/2 Running 0 31m
collector-g2g9c 2/2 Running 0 31m
collector-hcwzr 2/2 Running 0 31m
collector-kcngf 2/2 Running 0 40m
collector-km4rb 2/2 Running 0 40m
collector-stvng 2/2 Running 0 40m
collector-svb76 2/2 Running 0 40m
collector-zhwrp 2/2 Running 0 40m
elasticsearch-cdm-7fwy2cun-1-7c74465cc7-f98l9 2/2 Running 0 34m
elasticsearch-cdm-7fwy2cun-2-6d4f77b8b8-hn8bg 2/2 Running 0 34m
elasticsearch-cdm-7fwy2cun-3-7b56b6cccf-wfx4c 2/2 Running 0 34m
elasticsearch-im-app-27958455-dhwjr 0/1 Completed 0 3m24s
elasticsearch-im-audit-27958455-4c4mj 0/1 Completed 0 3m24s
elasticsearch-im-infra-27958455-8lxfk 0/1 Completed 0 3m24s
kibana-7b75b667-c8z44 2/2 Running 0 41m
logging-loki-compactor-0 1/1 Running 0 59m
logging-loki-distributor-7b8dbc758f-nhhgc 1/1 Running 0 19m
logging-loki-gateway-89d6d5ffc-dsk6f 2/2 Running 0 59m
logging-loki-gateway-89d6d5ffc-scvpz 2/2 Running 0 59m
logging-loki-index-gateway-0 1/1 Running 0 59m
logging-loki-ingester-0 1/1 Running 0 59m
logging-loki-querier-77b5f46675-9xzqx 1/1 Running 0 59m
logging-loki-query-frontend-6447bd9796-rstms 1/1 Running 0 59m
Attachments
Issue Links
- clones
-
-
- Closed
-
- links to
- mentioned on
