Major Description of problem:
Missing secret after elasticsearch or loki operator upgrade.
After elasticsearch operator upgrade secret "elasticsearch-operator-metrics-token" is missing automatically and below logs are present in the prometheus operator pod:
level=warn ts=2024-05-02T06:39:17.996325112Z caller=operator.go:2255 component=prometheusoperator msg="skipping servicemonitor" error="failed to get CA: unable to get secret \"elasticsearch-operator-metrics-token\": secrets \"elasticsearch-operator-metrics-token\" not found" servicemonitor=openshift-operators-redhat/elasticsearch-operator-metrics-monitor namespace=openshift-monitoring prometheus=k8s
After loki operator upgrade secret "loki-operator-controller-manager-metrics-token" is missing automatically and below logs are present in the prometheus operator pod:
level=warn ts=2024-03-29T13:41:15.373175003Z caller=operator.go:1917 component=prometheusoperator msg="skipping servicemonitor" error="failed to get CA: unable to get secret \"loki-operator-controller-manager-metrics-token\": secrets \"loki-operator-controller-manager-metrics-token\" not found" servicemonitor=openshift-operators-redhat/loki-operator-metrics-monitor namespace=openshift-monitoring prometheus=k8s
How Reproducible:
Not 100% reproducible.
Steps to Reproduce:
Upgrade loki / elasticsearch operator. Issue faced on: - elasticsearch operator upgrade to version 5.6.18 and 5.8.6 - loki to 5.8.5 and 5.9.0.
Actual results:
Secret "elasticsearch-operator-metrics-token" or "loki-operator-controller-manager-metrics-token" does not get recreated automatically on operator upgrade respectively.
Expected results:
Secret "elasticsearch-operator-metrics-token" or "loki-operator-controller-manager-metrics-token" should get recreated automatically on operator upgrade respectively.