[Logging 5.4] Kibana pod can't connect to ES cluster after removing secret/signing-elasticsearch: "x509: certificate signed by unknown authority"

Description of problem:

Removing secret/signing-elasticsearch and waiting for the secret and ES cluster back to normal, try to log into kibana console, the kibana can't connect to ES. Checking logs in ES pod, many error logs in  proxy container:

2022/03/10 01:32:07 http: TLS handshake error from 10.128.2.20:60856: tls: failed to verify client certificate: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "Logging Signing CA")
2022/03/10 01:32:12 http: TLS handshake error from 10.128.2.20:60910: tls: failed to verify client certificate: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "Logging Signing CA") 

In kibana console, it shows:

plugin:elasticsearch@6.8.1 Unable to connect to Elasticsearch. 

Version-Release number of selected component (if applicable):

EO image: quay.io/openshift-logging/elasticsearch-operator@sha256:5129b399aae941ba57284556a655774d0d62e6881850e49dbb32cf10c4955829

How reproducible:

Always

Steps to Reproduce:

1. deploy logging
2. remove secret/signing-elasticsearch
3. wait for the secret to be recreated 
4. wait for ES pods to be up and running, log into kibana console

Actual results:

Kibana can't connect to ES

Expected results:

Additional info: 

Workaround: remove secret/kibana and wait for EO to recreate secret/kibana and pod/kibana