* OCP: 4.7.24
* cluster-logging version 5.1.0-75
* OpenShift Elasticsearch Operator 5.1.0-96

2021-08-22T14:15:16.936670135Z Checking if infra-000225 is the write index for infra-write
2021-08-22T14:15:16.988613680Z Done!
2021-08-22T14:15:16.990845795Z ========================
2021-08-22T14:15:16.990845795Z Index management rollover process starting for {"error":{"root_cause":[{"type":"security_exception","reason":"_opendistro_security_dls_query
2021-08-22T14:15:16.990845795Z 
2021-08-22T14:15:16.999455040Z Received an empty response from elasticsearch -- server may not be readyChecking the ES instances for messages caused by the request:

% for pod in $(oc get pods -n openshift-logging|awk '/-cdm-/ {print $1}'); do echo $pod; oc logs pod/$pod -n openshift-logging -c elasticsearch |grep "2021-08-22T14:15:1";  done
elasticsearch-cdm-u27cy1z5-1-75ffb9c455-k9b7p
elasticsearch-cdm-u27cy1z5-2-6d4668f58-9s9dr
elasticsearch-cdm-u27cy1z5-3-745687cf85-fdf9q
2021-08-22T14:15:11.787176744Z [2021-08-22T14:15:11,669][WARN ][r.suppressed             ] [elasticsearch-cdm-u27cy1z5-3] path: /audit-write/_rollover, params: {pretty=, index=audit-write}
2021-08-22T14:15:11.787176744Z org.elasticsearch.transport.RemoteTransportException: [elasticsearch-cdm-u27cy1z5-2][10.131.2.18:9300][indices:admin/rollover]
2021-08-22T14:15:11.787176744Z Caused by: org.elasticsearch.ElasticsearchSecurityException: _opendistro_security_dls_query does not match (SG 900D)
2021-08-22T14:15:11.787176744Z     at com.amazon.opendistroforelasticsearch.security.privileges.DlsFlsEvaluator.evaluate(DlsFlsEvaluator.java:128) ~[?:?]
2021-08-22T14:15:11.787176744Z     at com.amazon.opendistroforelasticsearch.security.privileges.PrivilegesEvaluator.evaluate(PrivilegesEvaluator.java:443) ~[?:?]
2021-08-22T14:15:11.787176744Z     at com.amazon.opendistroforelasticsearch.security.filter.OpenDistroSecurityFilter.apply0(OpenDistroSecurityFilter.java:287) ~[?:?]
...
2021-08-22T14:15:11.787410641Z [2021-08-22T14:15:11,669][WARN ][r.suppressed             ] [elasticsearch-cdm-u27cy1z5-3] path: /_cat/aliases/infra-write, params: {h=alias, alias=infra-write}
2021-08-22T14:15:11.787410641Z org.elasticsearch.transport.RemoteTransportException: [elasticsearch-cdm-u27cy1z5-2][10.131.2.18:9300][indices:admin/aliases/get]
2021-08-22T14:15:11.787410641Z Caused by: org.elasticsearch.ElasticsearchSecurityException: _opendistro_security_dls_query does not match (SG 900D)
2021-08-22T14:15:11.787410641Z     at com.amazon.opendistroforelasticsearch.security.privileges.DlsFlsEvaluator.evaluate(DlsFlsEvaluator.java:128) ~[?:?]
2021-08-22T14:15:11.787410641Z     at com.amazon.opendistroforelasticsearch.security.privileges.PrivilegesEvaluator.evaluate(PrivilegesEvaluator.java:443) ~[?:?]