-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
4.17.0
-
None
-
Low
-
None
-
False
-
Description of problem:
"Failed to remove file watch, it may have been deleted" file="/etc/tls/client/client-ca.crt" err="can't remove non-existent watcher: /etc/tls/client/client-ca.crt" logs in kube-rbac-proxy/kube-rbac-proxy-main/kube-rbac-proxy-self container logs for some pods, include kube-state-metrics/node-exporter/openshift-state-metrics/prometheus-operator/telemeter-client pods, no functionality affected, the file "/etc/tls/client/client-ca.crt" exists.
$ for project in openshift-monitoring; do echo "project is: $project"; for pod in $(oc -n $project get pod --no-headers | awk '{print $1}'); do echo "pod is $pod"; for container in $(oc -n $project get pods $pod -o jsonpath="{.spec.containers[*].name}"); do echo "container: $container"; oc -n $project logs -c $container $pod | grep "Failed to remove file watch" | head -n 2;echo -e "\n"; done; echo -e "\n"; done; done
...
pod is kube-state-metrics-db5bd95-szjw6
container: kube-state-metrics
container: kube-rbac-proxy-main
I0728 23:26:00.459861 1 dynamic_cafile_content.go:211] "Failed to remove file watch, it may have been deleted" file="/etc/tls/client/client-ca.crt" err="can't remove non-existent watcher: /etc/tls/client/client-ca.crt"
container: kube-rbac-proxy-self
I0728 23:26:00.459867 1 dynamic_cafile_content.go:211] "Failed to remove file watch, it may have been deleted" file="/etc/tls/client/client-ca.crt" err="can't remove non-existent watcher: /etc/tls/client/client-ca.crt"
...
pod is node-exporter-9gwkl
container: node-exporter
container: kube-rbac-proxy
I0728 23:25:56.877374 5797 dynamic_cafile_content.go:211] "Failed to remove file watch, it may have been deleted" file="/etc/tls/client/client-ca.crt" err="can't remove non-existent watcher: /etc/tls/client/client-ca.crt"
...
pod is openshift-state-metrics-58956c78d5-g876s
container: kube-rbac-proxy-main
I0728 23:25:52.511328 1 dynamic_cafile_content.go:211] "Failed to remove file watch, it may have been deleted" file="/etc/tls/client/client-ca.crt" err="can't remove non-existent watcher: /etc/tls/client/client-ca.crt"
container: kube-rbac-proxy-self
I0728 23:25:52.511336 1 dynamic_cafile_content.go:211] "Failed to remove file watch, it may have been deleted" file="/etc/tls/client/client-ca.crt" err="can't remove non-existent watcher: /etc/tls/client/client-ca.crt"
...
pod is prometheus-operator-b8c7dfcfb-kbxww
container: prometheus-operator
container: kube-rbac-proxy
I0728 23:26:47.666705 1 dynamic_cafile_content.go:211] "Failed to remove file watch, it may have been deleted" file="/etc/tls/client/client-ca.crt" err="can't remove non-existent watcher: /etc/tls/client/client-ca.crt"
pod is telemeter-client-7bb8c547c7-77gwn
container: telemeter-client
container: reload
container: kube-rbac-proxy
I0728 23:26:14.445626 1 dynamic_cafile_content.go:211] "Failed to remove file watch, it may have been deleted" file="/etc/tls/client/client-ca.crt" err="can't remove non-existent watcher: /etc/tls/client/client-ca.crt"
...
the file exists, example
$ oc -n openshift-monitoring exec -c kube-rbac-proxy-main kube-state-metrics-db5bd95-szjw6 -- ls /etc/tls/client/client-ca.crt -al lrwxrwxrwx. 1 root 1000440000 20 Jul 28 23:20 /etc/tls/client/client-ca.crt -> ..data/client-ca.crt $ oc -n openshift-monitoring exec -c kube-rbac-proxy-self openshift-state-metrics-58956c78d5-g876s -- ls /etc/tls/client/client-ca.crt -al lrwxrwxrwx. 1 root 1000440000 20 Jul 28 23:20 /etc/tls/client/client-ca.crt -> ..data/client-ca.crt
Version-Release number of selected component (if applicable):
4.17.0-0.nightly-2024-07-28-142625
How reproducible:
always
Steps to Reproduce:
1. check pod logs
Actual results:
"Failed to remove file watch, it may have been deleted" message in kube-rbac-proxy container logs
Expected results:
no such message
Additional info:
