-
Bug
-
Resolution: Duplicate
-
Undefined
-
None
-
Logging 5.9.z, Logging 6.2.z
-
None
-
False
-
-
False
-
NEW
-
NEW
-
Bug Fix
-
-
-
Log Storage - Sprint 269
-
Moderate
Description of problem:
When it's created a second LokiStack CR with the same name that one existing in the `openshift-logging` some errors are visible.
LokiStack CR:
$ oc get lokistack -n openshift-logging NAME AGE logging-loki 18h $ oc get lokistack -n test-loki NAME AGE logging-loki 14m
If the creation of this second instance with the same name was not allowed, it should be expected:
1. The LokiStack Operator didn't allow not creating the resources for avoiding the current conflict
2. Also documentation should document the scenario not allowed
Then, from the point of view of an OpenShift Admin, the configuration done is something supported and valid failing as a bug exists.
Errors:
The LokiStack Gateway pods have errors of `Unauthorized` as not able to created the `tokenreviews`:
$ oc logs logging-loki-gateway-5fd9f59b7b-4jmt4 -n openshift-logging [...] level=error name=lokistack-gateway ts=2025-04-15T07:25:49.770333877Z caller=opa.go:159 tenant=audit msg="received non-200 status code from OPA endpoint" URL=[http://localhost:8082/v1/data/lokistack/allow] body="cluster-wide SAR failed: failed to create subject access review: subjectaccessreviews.authorization.k8s.io is forbidden: User \"system:serviceaccount:openshift-logging:logging-loki-gateway\" cannot create resource \"subjectaccessreviews\" in API group \"authorization.k8s.io\" at the cluster scope\n" status="401 Unauthorized" E0415 07:25:50.011743 1 webhook.go:154] Failed to make webhook authenticator request: tokenreviews.authentication.k8s.io is forbidden: User "system:serviceaccount:openshift-logging:logging-loki-gateway" cannot create resource "tokenreviews" in API group "authentication.k8s.io" at the cluster scope [...]
If the `logging-loki-gateway` pods are restarted, then, it starts to work as able to create the `tokenreviews`, but it will fail again after some minutes.
As consequence of the LokiStack Gateway pods not able to authenticate:
1. the logs in the OpenShift Console and the error "alert:JSON.parse: unexpected character at line 1 column 1 of the JSON data" appears
2. The collectors are not able to log forward to the LokiStack being observed the error:
2025-04-15T07:39:17.795469Z ERROR sink\{component_kind="sink" component_id=output_default_lokistack_infrastructure component_type=loki}: vector_common::internal_event::service: Service call failed. No retries or retries exhausted. error=Some(ServerError \{ code: 302 }) request_id=28397 error_type="request_failed" stage="sending" internal_log_rate_limit=true
Version-Release number of selected component (if applicable):
Tested in Logging 5.9 and 6.2, but expected to fail in other versions
How reproducible:
Always
Steps to Reproduce:
1. Deploy a Lokistack CR called `logging-loki` in the OpenShift Logging namespace
2. Create a namespace, for instance, `test-loki` and deploy a second LokiStack CR called also `logging-loki`
3. Install the Red Hat Cluster Logging Operator and configure the clusterLogForwarder to send the logs to the LokiStack CR `logging-loki` in the namespace `openshift-logging`
Actual results:
The LokiStack Gateway pods have errors of `Unauthorized` as not able to created the `tokenreviews`:
$ oc logs logging-loki-gateway-5fd9f59b7b-4jmt4 -n openshift-logging [...] level=error name=lokistack-gateway ts=2025-04-15T07:25:49.770333877Z caller=opa.go:159 tenant=audit msg="received non-200 status code from OPA endpoint" URL=[http://localhost:8082/v1/data/lokistack/allow] body="cluster-wide SAR failed: failed to create subject access review: subjectaccessreviews.authorization.k8s.io is forbidden: User \"system:serviceaccount:openshift-logging:logging-loki-gateway\" cannot create resource \"subjectaccessreviews\" in API group \"authorization.k8s.io\" at the cluster scope\n" status="401 Unauthorized" E0415 07:25:50.011743 1 webhook.go:154] Failed to make webhook authenticator request: tokenreviews.authentication.k8s.io is forbidden: User "system:serviceaccount:openshift-logging:logging-loki-gateway" cannot create resource "tokenreviews" in API group "authentication.k8s.io" at the cluster scope [...]
If the `logging-loki-gateway` pods are restarted, then, it starts to work as able to create the `tokenreviews`, but it will fail again after some minutes.
As consequence of the LokiStack Gateway pods not able to authenticate:
1. the logs in the OpenShift Console and the error "alert:JSON.parse: unexpected character at line 1 column 1 of the JSON data" appears
2. The collectors are not able to log forward to the LokiStack being observed the error:
2025-04-15T07:39:17.795469Z ERROR sink\{component_kind="sink" component_id=output_default_lokistack_infrastructure component_type=loki}: vector_common::internal_event::service: Service call failed. No retries or retries exhausted. error=Some(ServerError \{ code: 302 }) request_id=28397 error_type="request_failed" stage="sending" internal_log_rate_limit=true
Expected results:
The creation of a second LokiStack CR with the same name that one existing doesn't affect to the functionality or if the creation of a second LokiStack CR with the same name was not allowed, it should be expected:
1. The LokiStack Operator didn't allow not creating the resources for avoiding the current conflict
2. Also documentation should document the scenario not allowed
Additional info:
- duplicates
-
-
- POST
-
- links to
