[4.13] static container pod cannot be running due to CNI request failed with status 400

static container pod cannot be running due to CNI request failed with status 400

4.13.0-0.nightly-2022-12-19-122634

Always

Steps to Reproduce:
1. Create pod yaml file in /etc/kubernetes/manifests/ of worker $oc debug node/worker1sh-4.4# cd /etc/kubernetes/manifests/
sh-4.4# cat static-web.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: static-web
  namespace: zhouyt
  labels:
    role: myrole
spec:
  containers:
    - name: web
      image: nginx
      ports:
        - name: web
          containerPort: 80
          protocol: TCP2. Check the pod cannot be running
3. oc describe pod 
 

[weliang@weliang networking]$ oc get pod
NAME                                                                  READY   STATUS              RESTARTS   AGE
static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal   0/1     ContainerCreating   0          12s
weliang-19b-g9qzt-worker-c-wbcw8copenshift-qeinternal-debug           1/1     Running             0          72s
[weliang@weliang networking]$ oc describe pod static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal
Name:           static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal
Namespace:      test
Priority:       0
Node:           weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal/10.0.128.4
Start Time:     Mon, 19 Dec 2022 15:01:19 -0500
Labels:         role=myrole
Annotations:    k8s.ovn.org/pod-networks:
                  {"default":{"ip_addresses":["10.129.2.10/23"],"mac_address":"0a:58:0a:81:02:0a","gateway_ips":["10.129.2.1"],"ip_address":"10.129.2.10/23"...
                kubernetes.io/config.hash: 6a73fd271f8cd9ce565febcfadd243ff
                kubernetes.io/config.mirror: 6a73fd271f8cd9ce565febcfadd243ff
                kubernetes.io/config.seen: 2022-12-19T20:01:19.483843942Z
                kubernetes.io/config.source: file
                openshift.io/scc: restricted-v2
                seccomp.security.alpha.kubernetes.io/pod: runtime/default
Status:         Pending
IP:             
IPs:            <none>
Controlled By:  Node/weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal
Containers:
  web:
    Container ID:   
    Image:          nginx
    Image ID:       
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:         <none>
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:            <none>
QoS Class:          BestEffort
Node-Selectors:     <none>
Tolerations:        :NoExecute op=Exists
Events:
  Type     Reason                  Age   From     Message
  ----     ------                  ----  ----     -------
  Warning  FailedCreatePodSandBox  41s   kubelet  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal_test_6a73fd271f8cd9ce565febcfadd243ff_0(5df7795e95cb798676d6b4717425bcc71d34d3bf6df6c2f56dbc42fac1046f37): error adding pod test_static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): [test/static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal/6a73fd271f8cd9ce565febcfadd243ff:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[test/static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal 5df7795e95cb798676d6b4717425bcc71d34d3bf6df6c2f56dbc42fac1046f37] [test/static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal 5df7795e95cb798676d6b4717425bcc71d34d3bf6df6c2f56dbc42fac1046f37] pod deleted before sandbox ADD operation began
'
  Warning  FailedCreatePodSandBox  40s  kubelet  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal_test_6a73fd271f8cd9ce565febcfadd243ff_0(4784df8acd72f7b93d62318e18eed337ae6c30d478c742e220832ab4b20c8f06): error adding pod test_static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): [test/static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal/6a73fd271f8cd9ce565febcfadd243ff:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[test/static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal 4784df8acd72f7b93d62318e18eed337ae6c30d478c742e220832ab4b20c8f06] [test/static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal 4784df8acd72f7b93d62318e18eed337ae6c30d478c742e220832ab4b20c8f06] pod deleted before sandbox ADD operation began
'
  Warning  FailedCreatePodSandBox  24s  kubelet  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal_test_6a73fd271f8cd9ce565febcfadd243ff_0(2dc27f2f6b0cbed977ae469d7e5a62ef678705edc8ed81fe1b265caee40604fb): error adding pod test_static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): [test/static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal/6a73fd271f8cd9ce565febcfadd243ff:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[test/static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal 2dc27f2f6b0cbed977ae469d7e5a62ef678705edc8ed81fe1b265caee40604fb] [test/static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal 2dc27f2f6b0cbed977ae469d7e5a62ef678705edc8ed81fe1b265caee40604fb] pod deleted before sandbox ADD operation began
'
  Warning  FailedCreatePodSandBox  8s  kubelet  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal_test_6a73fd271f8cd9ce565febcfadd243ff_0(1e708dffbbe728e02422fae7ecb5dcb7c21891b20d5f595f0f1d56bea6b37417): error adding pod test_static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): [test/static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal/6a73fd271f8cd9ce565febcfadd243ff:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[test/static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal 1e708dffbbe728e02422fae7ecb5dcb7c21891b20d5f595f0f1d56bea6b37417] [test/static-web-weliang-19b-g9qzt-worker-c-wbcw8.c.openshift-qe.internal 1e708dffbbe728e02422fae7ecb5dcb7c21891b20d5f595f0f1d56bea6b37417] pod deleted before sandbox ADD operation began
'
[weliang@weliang networking]$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.13.0-0.nightly-2022-12-19-122634   True        False         45m     Cluster version is 4.13.0-0.nightly-2022-12-19-122634
[weliang@weliang networking]$ 

Pod in Running state