-
Bug
-
Resolution: Won't Do
-
Major
-
None
-
4.14.z
-
None
-
Incidents & Support
-
False
-
-
None
-
Important
-
None
-
None
-
None
-
None
-
In Progress
-
Release Note Not Required
-
None
-
None
-
None
-
None
-
None
Description of problem:
We have seen the following error messages during the pod creation.
Jan 07 19:08:30 cocmsusgv102.sg.uobnet.com crio[73263]: 2025-01-07T19:08:30+08:00 [error] CmdCheck (shim): failed to send CNI request: Post "http://dummy/cni": dial unix /run/multus/socket/multus.sock: connect: connection refused --- Jan 07 19:35:53 cocmsusgv102.sg.uobnet.com crio[669104]: 2025-01-07T19:35:53+08:00 [error] CmdCheck (shim): CNI request failed with status 400: '&{ContainerID:7c1221ae2553965b24a2d8fa050c29ca13568b16e41ce10b3b2cacbb80e91503 Netns:/var/run/netns/a21b1a79-65dc-4484-a0d9-50f3c5f155bd IfName:eth0 Args:IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-kube-apiserver;K8S_POD_NAME=installer-366-cocmsusgv102.sg.uobnet.com;K8S_POD_INFRA_CONTAINER_ID=7c1221ae2553965b24a2d8fa050c29ca13568b16e41ce10b3b2cacbb80e91503;K8S_POD_UID=acaf4850-c013-4337-9646-f120f1bcf5c5 Path: StdinData:[123 34 98 105 110 68 105 114 34 58 34 47 118 97 114 47 108 105 98 47 99 110 105 47 98 105 110 34 44 34 99 104 114 111 111 116 68 105 114 34 58 34 47 104 111 115 116 114 111 111 116 34 44 34 99 108 117 115 116 101 114 78 101 116 119 111 114 107 34 58 34 47 104 111 115 116 47 114 117 110 47 109 117 108 116 117 115 47 99 110 105 47 110 101 116 46 100 47 56 48 45 111 112 101 110 115 104 105 102 116 45 110 101 116 119 111 114 107 46 99 111 110 102 34 44 34 99 110 105 67 111 110 102 105 103 68 105 114 34 58 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 34 44 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 51 46 49 34 44 34 100 97 101 109 111 110 83 111 99 107 101 116 68 105 114 34 58 34 47 114 117 110 47 109 117 108 116 117 115 47 115 111 99 107 101 116 34 44 34 103 108 111 98 97 108 78 97 109 101 115 112 97 99 101 115 34 58 34 100 101 102 97 117 108 116 44 111 112 101 110 115 104 105 102 116 45 109 117 108 116 117 115 44 111 112 101 110 115 104 105 102 116 45 115 114 105 111 118 45 110 101 116 119 111 114 107 45 111 112 101 114 97 116 111 114 34 44 34 108 111 103 76 101 118 101 108 34 58 34 118 101 114 98 111 115 101 34 44 34 108 111 103 84 111 83 116 100 101 114 114 34 58 116 114 117 101 44 34 109 117 108 116 117 115 65 117 116 111 99 111 110 102 105 103 68 105 114 34 58 34 47 104 111 115 116 47 114 117 110 47 109 117 108 116 117 115 47 99 110 105 47 110 101 116 46 100 34 44 34 109 117 108 116 117 115 67 111 110 102 105 103 70 105 108 101 34 58 34 97 117 116 111 34 44 34 110 97 109 101 34 58 34 109 117 108 116 117 115 45 99 110 105 45 110 101 116 119 111 114 107 34 44 34 110 97 109 101 115 112 97 99 101 73 115 111 108 97 116 105 111 110 34 58 116 114 117 101 44 34 112 101 114 78 111 100 101 67 101 114 116 105 102 105 99 97 116 101 34 58 123 34 98 111 111 116 115 116 114 97 112 75 117 98 101 99 111 110 102 105 103 34 58 34 47 118 97 114 47 108 105 98 47 107 117 98 101 108 101 116 47 107 117 98 101 99 111 110 102 105 103 34 44 34 99 101 114 116 68 105 114 34 58 34 47 101 116 99 47 99 110 105 47 109 117 108 116 117 115 47 99 101 114 116 115 34 44 34 99 101 114 116 68 117 114 97 116 105 111 110 34 58 34 50 52 104 34 44 34 101 110 97 98 108 101 100 34 58 116 114 117 101 125 44 34 115 111 99 107 101 116 68 105 114 34 58 34 47 104 111 115 116 47 114 117 110 47 109 117 108 116 117 115 47 115 111 99 107 101 116 34 44 34 116 121 112 101 34 58 34 109 117 108 116 117 115 45 115 104 105 109 34 125]} ContainerID:"7c1221ae2553965b24a2d8fa050c29ca13568b16e41ce10b3b2cacbb80e91503" Netns:"/var/run/netns/a21b1a79-65dc-4484-a0d9-50f3c5f155bd" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-kube-apiserver;K8S_POD_NAME=installer-366-cocmsusgv102.sg.uobnet.com;K8S_POD_INFRA_CONTAINER_ID=7c1221ae2553965b24a2d8fa050c29ca13568b16e41ce10b3b2cacbb80e91503;K8S_POD_UID=acaf4850-c013-4337-9646-f120f1bcf5c5" Path:"" ERRORED: DelegateDel: error invoking DelegateDel - "openshift-sdn": error in getting result from DelNetwork: failed to send CNI request: Post "http://dummy/": dial unix /var/run/openshift-sdn/cniserver/socket: connect: connection refused --- Jan 07 19:42:51 cocmsusgv102.sg.uobnet.com crio[1946]: 2025-01-07T19:42:51+08:00 [error] CmdCheck (shim): failed to send CNI request: Post "http://dummy/cni": dial unix /run/multus/socket/multus.sock: connect: no such file or directory --- Jan 07 20:06:40 cocmsusgv102.sg.uobnet.com kubenswrapper[1976]: E0107 20:06:40.461213 1976 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"controller-manager-78d9c5955d-nftps_openshift-controller-manager(44fde62c-03e1-48c7-a889-550f3fcedcee)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"controller-manager-78d9c5955d-nftps_openshift-controller-manager(44fde62c-03e1-48c7-a889-550f3fcedcee)\\\": rpc error: code = Unknown desc = failed to create pod network sandbox k8s_controller-manager-78d9c5955d-nftps_openshift-controller-manager_44fde62c-03e1-48c7-a889-550f3fcedcee_0(b5757802fd01f996ffc1ca633a838b318335e868964322fccef8127067c9ff26): error adding pod openshift-controller-manager_controller-manager-78d9c5955d-nftps to CNI network \\\"multus-cni-network\\\": plugin type=\\\"multus-shim\\\" name=\\\"multus-cni-network\\\" failed (add): CmdAdd (shim): failed to send CNI request: Post \\\"http://dummy/cni\\\": EOF\"" pod="openshift-controller-manager/controller-manager-78d9c5955d-nftps" podUID=44fde62c-03e1-48c7-a889-550f3fcedcee
Version-Release number of selected component (if applicable):
How reproducible:
Steps to Reproduce:
1.
2.
3.
Actual results:
Expected results:
Additional info:
Please fill in the following template while reporting a bug and provide as much relevant information as possible. Doing so will give us the best chance to find a prompt resolution.
Affected Platforms:
Is it an
- internal CI failure
- customer issue / SD
- internal RedHat testing failure
If it is an internal RedHat testing failure:
- Please share a kubeconfig or creds to a live cluster for the assignee to debug/troubleshoot along with reproducer steps (specially if it's a telco use case like ICNI, secondary bridges or BM+kubevirt).
If it is a CI failure:
- Did it happen in different CI lanes? If so please provide links to multiple failures with the same error instance
- Did it happen in both sdn and ovn jobs? If so please provide links to multiple failures with the same error instance
- Did it happen in other platforms (e.g. aws, azure, gcp, baremetal etc) ? If so please provide links to multiple failures with the same error instance
- When did the failure start happening? Please provide the UTC timestamp of the networking outage window from a sample failure run
- If it's a connectivity issue,
- What is the srcNode, srcIP and srcNamespace and srcPodName?
- What is the dstNode, dstIP and dstNamespace and dstPodName?
- What is the traffic path? (examples: pod2pod? pod2external?, pod2svc? pod2Node? etc)
If it is a customer / SD issue:
- Provide enough information in the bug description that Engineering doesn't need to read the entire case history.
- Don't presume that Engineering has access to Salesforce.
- Do presume that Engineering will access attachments through supportshell.
- Describe what each relevant attachment is intended to demonstrate (failed pods, log errors, OVS issues, etc).
- Referring to the attached must-gather, sosreport or other attachment, please provide the following details:
- If the issue is in a customer namespace then provide a namespace inspect.
- If it is a connectivity issue:
- What is the srcNode, srcNamespace, srcPodName and srcPodIP?
- What is the dstNode, dstNamespace, dstPodName and dstPodIP?
- What is the traffic path? (examples: pod2pod? pod2external?, pod2svc? pod2Node? etc)
- Please provide the UTC timestamp networking outage window from must-gather
- Please provide tcpdump pcaps taken during the outage filtered based on the above provided src/dst IPs
- If it is not a connectivity issue:
- Describe the steps taken so far to analyze the logs from networking components (cluster-network-operator, OVNK, SDN, openvswitch, ovs-configure etc) and the actual component where the issue was seen based on the attached must-gather. Please attach snippets of relevant logs around the window when problem has happened if any.
- When showing the results from commands, include the entire command in the output.
- For OCPBUGS in which the issue has been identified, label with "sbr-triaged"
- For OCPBUGS in which the issue has not been identified and needs Engineering help for root cause, label with "sbr-untriaged"
- Do not set the priority, that is owned by Engineering and will be set when the bug is evaluated
- Note: bugs that do not meet these minimum standards will be closed with label "SDN-Jira-template"
- For guidance on using this template please see
OCPBUGS Template Training for Networking components
- depends on
-
-
- Closed
-
- is cloned by
-
-
- Closed
-
- links to
