-
Bug
-
Resolution: Done
-
Major
-
None
-
None
-
None
-
False
-
-
False
-
-
-
Tracing Sprint # 284 - 3.9
-
Critical
Description of problem:
TempoStack version upgrade from 0.16.0-2 to 0.19.0-3 creates networkPolicies that break communication with Kubernetes API and the pods in openshift-tempo-operator namespace remain in crashloopbackoff state.
Once the policies are removed the openshift-tempo-operator namespace pods keep running normal for sometime until the operator again creates the NetworkPolicies back.
Error logs from the openshift-tempo-operator pod :
{"level":"error","ts":"2026-02-05T17:14:35.378522Z","logger":"setup","msg":"unable to create controller","controller":"TempoStack","error":"failed to get server groups: Get \"https://172.x.x.x:443/api\": dial tcp 172.x.x.x:443: i/o timeout","stacktrace":"github.com/grafana/tempo-operator/cmd/start.start\n\tgithub.com/grafana/tempo-operator/cmd/start/main.go:81\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/cobra@v1.9.1/command.go:1019\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/cobra@v1.9.1/command.go:1148\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/cobra@v1.9.1/command.go:1071\nmain.main\n\t./main.go:26\nruntime.main\n\truntime/proc.go:285"}
Version-Release number of selected component (if applicable):
TempoStack version : 0.19.0-3 Openshift version : 4.17
How reproducible:
Follow the below steps :
Steps to Reproduce:
1. Install the tempoStack operator with version 0.16.0-2
2. Upgrade the tempoStack operator to version 0.19.0-3 and create required instances for it.
3. See if 4 networkPolicies are created in the openshift-tempo-operator namespace :
tempo-operator-deny-all
tempo-operator-egress-to-apiserver
tempo-operator-ingress-to-metrics
tempo-operator-ingress-webhook
4. Check if the pods are in crashloopbackoff state in the same namespace
5. Check for the error logs from the openshift-tempo-operator pod.
6. Delete the networkPolicies from there and check state of pod.
Actual results:
The openshift-tempo-operator pod remain in crashloopbackoff state with it unable to communicate to the kubernetes api service.
Expected results:
The openshift-tempo-operator pod should be running even if the netPol are by default created.
Additional info: