-
Bug
-
Resolution: Duplicate
-
Critical
-
None
-
ACM 2.9.0
-
None
-
False
-
None
-
False
-
-
-
No
Description of problem:
ACM 2.9/ Submariner 0.16.0
The submariner-metrics-proxy pod crash on startup.
Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 160m default-scheduler Successfully assigned submariner-operator/submariner-metrics-proxy-5lggc to ip-10-0-64-117.us-east-2.compute.internal Warning ErrorUpdatingResource 160m (x3 over 160m) controlplane addLogicalPort failed for submariner-operator/submariner-metrics-proxy-5lggc: unable to parse node L3 gw annotation: k8s.ovn.org/l3-gateway-config annotation not found for node "ip-10-0-64-117.us-east-2.compute.internal" Warning NetworkNotReady 159m (x25 over 160m) kubelet network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started? Normal AddedInterface 159m multus Add eth0 [10.130.2.4/23] from ovn-kubernetes Warning Failed 75m kubelet Error: container create failed: time="2023-08-07T10:01:59Z" level=error msg="runc create failed: unable to start container process: exec: \"/app/metricsproxy\": permission denied" Warning Failed 40m kubelet Error: container create failed: time="2023-08-07T10:36:59Z" level=error msg="runc create failed: unable to start container process: exec: \"/app/metricsproxy\": permission denied" Normal Pulled 15m (x655 over 158m) kubelet Container image "registry.redhat.io/rhacm2/nettest-rhel8@sha256:0fab128d557b8803061846fdb70366a3b93d873648fc187eb9cc31471b67956e" already present on machine Warning Failed 10m kubelet Error: container create failed: time="2023-08-07T11:06:59Z" level=error msg="runc create failed: unable to start container process: exec: \"/app/metricsproxy\": permission denied" Warning Failed 5m8s kubelet Error: container create failed: time="2023-08-07T11:11:59Z" level=error msg="runc create failed: unable to start container process: exec: \"/app/metricsproxy\": permission denied" Warning Failed 8s kubelet Error: container create failed: time="2023-08-07T11:16:59Z" level=error msg="runc create failed: unable to start container process: exec: \"/app/metricsproxy\": permission denied"
Pod describe:
Name: submariner-metrics-proxy-5lggc Namespace: submariner-operator Priority: 0 Node: ip-10-0-64-117.us-east-2.compute.internal/10.0.64.117 Start Time: Mon, 07 Aug 2023 11:36:59 +0300 Labels: app=submariner-metrics-proxy component=metrics controller-revision-hash=86447d97f6 pod-template-generation=1 Annotations: k8s.ovn.org/pod-networks: {"default":{"ip_addresses":["10.130.2.4/23"],"mac_address":"0a:58:0a:82:02:04","gateway_ips":["10.130.2.1"],"ip_address":"10.130.2.4/23","... k8s.v1.cni.cncf.io/network-status: [{ "name": "ovn-kubernetes", "interface": "eth0", "ips": [ "10.130.2.4" ], "mac": "0a:58:0a:82:02:04", "default": true, "dns": {} }] openshift.io/scc: restricted-v2 seccomp.security.alpha.kubernetes.io/pod: runtime/default Status: Pending IP: 10.130.2.4 IPs: IP: 10.130.2.4 Controlled By: DaemonSet/submariner-metrics-proxy Containers: gateway-metrics-proxy: Container ID: Image: registry.redhat.io/rhacm2/nettest-rhel8@sha256:0fab128d557b8803061846fdb70366a3b93d873648fc187eb9cc31471b67956e Image ID: Port: <none> Host Port: <none> Command: /app/metricsproxy Args: 8080 $(NODE_IP) 32780 State: Waiting Reason: CreateContainerError Ready: False Restart Count: 0 Environment: NODE_IP: (v1:status.hostIP) Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-96wmj (ro) globalnet-metrics-proxy: Container ID: Image: registry.redhat.io/rhacm2/nettest-rhel8@sha256:0fab128d557b8803061846fdb70366a3b93d873648fc187eb9cc31471b67956e Image ID: Port: <none> Host Port: <none> Command: /app/metricsproxy Args: 8081 $(NODE_IP) 32781 State: Waiting Reason: CreateContainerError Ready: False Restart Count: 0 Environment: NODE_IP: (v1:status.hostIP) Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-96wmj (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: kube-api-access-96wmj: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true ConfigMapName: openshift-service-ca.crt ConfigMapOptional: <nil> QoS Class: BestEffort Node-Selectors: submariner.io/gateway=true Tolerations: op=Exists node.kubernetes.io/disk-pressure:NoSchedule op=Exists node.kubernetes.io/memory-pressure:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists node.kubernetes.io/pid-pressure:NoSchedule op=Exists node.kubernetes.io/unreachable:NoExecute op=Exists node.kubernetes.io/unschedulable:NoSchedule op=Exists Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 160m default-scheduler Successfully assigned submariner-operator/submariner-metrics-proxy-5lggc to ip-10-0-64-117.us-east-2.compute.internal Warning ErrorUpdatingResource 160m (x3 over 160m) controlplane addLogicalPort failed for submariner-operator/submariner-metrics-proxy-5lggc: unable to parse node L3 gw annotation: k8s.ovn.org/l3-gateway-config annotation not found for node "ip-10-0-64-117.us-east-2.compute.internal" Warning NetworkNotReady 159m (x25 over 160m) kubelet network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started? Normal AddedInterface 159m multus Add eth0 [10.130.2.4/23] from ovn-kubernetes Warning Failed 75m kubelet Error: container create failed: time="2023-08-07T10:01:59Z" level=error msg="runc create failed: unable to start container process: exec: \"/app/metricsproxy\": permission denied" Warning Failed 40m kubelet Error: container create failed: time="2023-08-07T10:36:59Z" level=error msg="runc create failed: unable to start container process: exec: \"/app/metricsproxy\": permission denied" Normal Pulled 15m (x655 over 158m) kubelet Container image "registry.redhat.io/rhacm2/nettest-rhel8@sha256:0fab128d557b8803061846fdb70366a3b93d873648fc187eb9cc31471b67956e" already present on machine Warning Failed 10m kubelet Error: container create failed: time="2023-08-07T11:06:59Z" level=error msg="runc create failed: unable to start container process: exec: \"/app/metricsproxy\": permission denied" Warning Failed 5m8s kubelet Error: container create failed: time="2023-08-07T11:11:59Z" level=error msg="runc create failed: unable to start container process: exec: \"/app/metricsproxy\": permission denied" Warning Failed 8s kubelet Error: container create failed: time="2023-08-07T11:16:59Z" level=error msg="runc create failed: unable to start container process: exec: \"/app/metricsproxy\": permission denied"
- is duplicated by
-
ACM-6769 [OCP 4.14] submariner-metrics-proxy pod in CreateContainerError state
-
- Closed
-