-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
4.17.z, 4.16.z, 4.18.z
This is a clone of issue OCPBUGS-54975. The following is the description of the original issue:
—
Description of problem:
Cinder CSI driver controller targets showing down for prometheus: ~~~ openstack-cinder-csi-driver-controller-854d987bc9-n4ztn https://10.129.0.48:9202/metrics down server returned HTTP status 502 Bad Gateway openstack-cinder-csi-driver-controller-854d987bc9-qkzlx https://10.130.0.38:9202/metrics down server returned HTTP status 502 Bad Gateway ~~~ Servicemonitor is scraping metrics on driver-m port: ~~~ apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor name: openstack-cinder-csi-driver-controller-monitor namespace: openshift-cluster-csi-drivers spec: endpoints: interval: 30s path: /metrics port: driver-m scheme: https ~~~ When the endpoints yaml are checked, driver-m port is mapped to 9202 port. ~~~ apiVersion: v1 kind: Endpoints name: openstack-cinder-csi-driver-controller-metrics namespace: openshift-cluster-csi-drivers subsets: - addresses: - ip: 10.129.0.48 nodeName: master-xvpnp-0 targetRef: kind: Pod name: openstack-cinder-csi-driver-controller-854d987bc9-n4ztn namespace: openshift-cluster-csi-drivers - ip: 10.130.0.38 nodeName: master-fd4q2-1 targetRef: kind: Pod name: openstack-cinder-csi-driver-controller-854d987bc9-qkzlx namespace: openshift-cluster-csi-drivers ports: - name: driver-m port: 9202 protocol: TCP ~~~ From pod yaml, 9202 port is mapped to container: kube-rbac-proxy-8202 ~~~ apiVersion: v1 kind: Pod name: openstack-cinder-csi-driver-controller-854d987bc9-n4ztn namespace: openshift-cluster-csi-drivers spec: containers: - args: - --secure-listen-address=0.0.0.0:9202 - --upstream=http://127.0.0.1:8202/ name: kube-rbac-proxy-8202 ports: - containerPort: 9202 name: driver-m protocol: TCP ~~~ This container is showing below logs: ~~~ 2025-04-10T08:27:22.374262574Z I0410 08:27:22.374224 1 kube-rbac-proxy.go:395] Starting TCP socket on 0.0.0.0:9202 2025-04-10T08:27:22.374502030Z I0410 08:27:22.374471 1 kube-rbac-proxy.go:402] Listening securely on 0.0.0.0:9202 2025-04-10T08:27:29.071617312Z I0410 08:27:29.071576 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8202: connect: connection refused 2025-04-10T08:27:46.671558739Z I0410 08:27:46.668164 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8202: connect: connection refused2025-04-11T09:26:59.064050080Z I0411 09:26:59.064000 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8202: connect: connection refused 2025-04-11T09:27:16.661091884Z I0411 09:27:16.659746 1 log.go:245] http: proxy error: dial tcp 127.0.0.1:8202: connect: connection refused ~~~ Other targets for same pod are showing up: ~~~ openstack-cinder-csi-driver-controller-854d987bc9-qkzlx https://10.130.0.38:9203/metrics up openstack-cinder-csi-driver-controller-854d987bc9-n4ztn https://10.129.0.48:9203/metrics up openstack-cinder-csi-driver-controller-854d987bc9-n4ztn https://10.129.0.48:9204/metrics up openstack-cinder-csi-driver-controller-854d987bc9-qkzlx https://10.130.0.38:9204/metrics up openstack-cinder-csi-driver-controller-854d987bc9-qkzlx https://10.130.0.38:9205/metrics up openstack-cinder-csi-driver-controller-854d987bc9-n4ztn https://10.129.0.48:9205/metrics up openstack-cinder-csi-driver-controller-854d987bc9-n4ztn https://10.129.0.48:9206/metrics up openstack-cinder-csi-driver-controller-854d987bc9-qkzlx https://10.130.0.38:9206/metrics up ~~~
Version-Release number of selected component (if applicable):
OCP 4.18, 4.17 ,4.16 running Cinder CSI driver on OpenStack ENV
How reproducible:
Steps to Reproduce:
1. 2. 3.
Actual results:
openstack-cinder-csi-driver-controller target is showing as down for prometheus.
Expected results:
openstack-cinder-csi-driver-controller target should be up for prometheus.
Additional info:
- clones
-
OCPBUGS-54975 openstack-cinder-csi-driver-controller-metrics prometheus target showing down for 9202 port
-
- Verified
-
- is blocked by
-
OCPBUGS-54975 openstack-cinder-csi-driver-controller-metrics prometheus target showing down for 9202 port
-
- Verified
-
- links to