This is a clone of issue OCPBUGS-44723. The following is the description of the original issue:
—
Description of problem:
After the upgrade to OpenShift Container Platform 4.17, it's being observed that aws-efs-csi-driver-controller-metrics/aws-efs-csi-driver-controller-metrics is reporting target down state. When checking the newly created Container one can find the below logs, that may explain the effect seen/reported. $ oc logs aws-efs-csi-driver-controller-5b8d5cfdf4-zwh67 -c kube-rbac-proxy-8211 W1119 07:53:10.249934 1 deprecated.go:66] ==== Removed Flag Warning ====================== logtostderr is removed in the k8s upstream and has no effect any more. =============================================== I1119 07:53:10.250382 1 kube-rbac-proxy.go:233] Valid token audiences: I1119 07:53:10.250431 1 kube-rbac-proxy.go:347] Reading certificate files I1119 07:53:10.250645 1 kube-rbac-proxy.go:395] Starting TCP socket on 0.0.0.0:9211 I1119 07:53:10.250944 1 kube-rbac-proxy.go:402] Listening securely on 0.0.0.0:9211 I1119 07:54:01.440714 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:54:19.860038 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:54:31.432943 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:54:49.852801 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:55:01.433635 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:55:19.853259 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:55:31.432722 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:55:49.852606 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:56:01.432707 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:56:19.853137 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:56:31.440223 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:56:49.856349 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:57:01.432528 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:57:19.853132 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:57:31.433104 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:57:49.852859 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:58:01.433321 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused I1119 07:58:19.853612 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8211: connect: connection refused
Version-Release number of selected component (if applicable):
OpenShift Container Platform 4.17
How reproducible:
Always
Steps to Reproduce:
1. Install OpenShift Container Platform 4.17 2. Install aws-efs-csi-driver-operator 3. Create efs.csi.aws.com CSIDriver object and wait for the aws-efs-csi-driver-controller to roll out.
Actual results:
The below Target Down Alert is being raised 50% of the aws-efs-csi-driver-controller-metrics/aws-efs-csi-driver-controller-metrics targets in Namespace openshift-cluster-csi-drivers namespace have been unreachable for more than 15 minutes. This may be a symptom of network connectivity issues, down nodes, or failures within these components. Assess the health of the infrastructure and nodes running these targets and then contact support.
Expected results:
The ServiceMonitor endpoint should be reachable and properly responding with the desired information to monitor the health of the component.
Additional info:
- clones
-
OCPBUGS-44723 aws-efs-csi-driver-controller-metrics/aws-efs-csi-driver-controller-metrics target down with OpenShift Container Platform 4.17
- Verified
- is blocked by
-
OCPBUGS-44723 aws-efs-csi-driver-controller-metrics/aws-efs-csi-driver-controller-metrics target down with OpenShift Container Platform 4.17
- Verified
- links to