Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-22101

Hosted Cluster pod creation failed with timed out waiting for OVS port binding

XMLWordPrintable

    • Important
    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      When we deploy new pods on an environement with Hosted Control plane I get the following error when I try to create new pod. 

       

      Version-Release number of selected component (if applicable):

      Openshift 4.13 / MCE 2.3 / Local Cloud Provider

      How reproducible:

      create a 4.13 stable hosted cluster on separate account on Outscale and scale with an important number of pod

      Steps to Reproduce:

      1.
      2.
      3.
      

      Actual results:

      Pod are in ContainerCreating state

      Expected results:

      Pod in running state

      Additional info:

      oc -n openshift-ovn-kubernetes logs ovnkube-node-t8h2q -c ovn-controller
      
      2023-10-19T08:20:19.147Z|05741|stream_ssl|WARN|SSL_connect: error:0A000126:SSL routines::unexpected eof while reading57432023-10-19T08:20:27.156Z|05742|stream_ssl|WARN|SSL_connect: error:0A000126:SSL routines::unexpected eof while reading57442023-10-19T08:20:35.159Z|05743|stream_ssl|WARN|SSL_connect: error:0A000126:SSL routines::unexpected eof while reading57452023-10-19T08:20:43.169Z|05744|stream_ssl|WARN|SSL_connect: error:0A000126:SSL routines::unexpected eof while reading57462023-10-19T08:20:51.181Z|05745|stream_ssl|WARN|SSL_connect: error:0A000126:SSL routines::unexpected eof while reading57472023-10-19T08:20:59.184Z|05746|stream_ssl|WARN|SSL_connect: error:0A000126:SSL routines::unexpected eof while reading
      
      
      oc -n openshift-ovn-kubernetes logs ovnkube-node-t8h2q -c ovn-kubenode
      
      W1018 20:56:18.216211 1471497 helper_linux.go:649] Failed to delete pod "open-cluster-management-agent-addon/klusterlet-addon-search-67d85457cd-cmtk2" interface b65cc98d4aeae1a: failed to lookup link b65cc98d4aeae1a: Link not found14423I1018 20:56:18.225953 1471497 cni.go:286] [open-cluster-management-agent-addon/klusterlet-addon-search-67d85457cd-cmtk2 b65cc98d4aeae1aad7cc75eada9f28567460307e59efef917418267aed605304] DEL finished CNI request [open-cluster-management-agent-addon/klusterlet-addon-search-67d85457cd-cmtk2 b65cc98d4aeae1aad7cc75eada9f28567460307e59efef917418267aed605304], result "{\"dns\":{}}", err <nil>14424I1018 20:56:21.716291 1471497 cni.go:265] [2test/example d6c8a25080d5ade225f3f56f12203250ad2aa9ecc6e49e433a4013f167176482] ADD starting CNI request [2test/example d6c8a25080d5ade225f3f56f12203250ad2aa9ecc6e49e433a4013f167176482]14425I1018 20:56:21.719173 1471497 helper_linux.go:363] ConfigureOVS: namespace: 2test, podName: example, network: default, NAD default, SandboxID: "d6c8a25080d5ade225f3f56f12203250ad2aa9ecc6e49e433a4013f167176482", UID: "ec92ef33-22c1-4404-a9e1-8c5f23c91f28", MAC: 0a:58:0a:84:00:1c, IPs: [10.132.0.28/23]14426W1018 20:56:23.883425 1471497 helper_linux.go:432] [cluster-density-0/cluster-density-5-d785dfd8-qmwfm 198b41e82453d92c2b7adb82f7421a39dd124df23867a4091011077b26cc4fe2] pod uid a8306c13-d97c-4f71-b430-f76896c13e98: timed out waiting for OVS port binding (ovn-installed) for 0a:58:0a:84:00:a4 [10.132.0.164/23]14427I1018 20:56:23.913234 1471497 cni.go:286] [cluster-density-0/cluster-density-5-d785dfd8-qmwfm 198b41e82453d92c2b7adb82f7421a39dd124df23867a4091011077b26cc4fe2] ADD finished CNI request [cluster-density-0/cluster-density-5-d785dfd8-qmwfm 198b41e82453d92c2b7adb82f7421a39dd124df23867a4091011077b26cc4fe2], result "", err failed to configure pod interface: timed out waiting for OVS port binding (ovn-installed) for 0a:58:0a:84:00:a4 [10.132.0.164/23]14428I1018 20:56:24.029562 1471497 cni.go:265] [cluster-density-0/cluster-density-5-d785dfd8-qmwfm 198b41e82453d92c2b7adb82f7421a39dd124df23867a4091011077b26cc4fe2] DEL starting CNI request [cluster-density-0/cluster-density-5-d785dfd8-qmwfm 198b41e82453d92c2b7adb82f7421a39dd124df23867a4091011077b26cc4fe2]14429W1018 20:56:24.036243 1471497 helper_linux.go:649] Failed to delete pod "cluster-density-0/cluster-density-5-d785dfd8-qmwfm" interface 198b41e82453d92: failed to lookup link 198b41e82453d92: Link not found14430I1018 20:56:24.046190 1471497 cni.go:286] [cluster-density-0/cluster-density-5-d785dfd8-qmwfm 198b41e82453d92c2b7adb82f7421a39dd124df23867a4091011077b26cc4fe2] DEL finished CNI request [cluster-density-0/cluster-density-5-d785dfd8-qmwfm 198b41e82453d92c2b7adb82f7421a39dd124df23867a4091011077b26cc4fe2], result "{\"dns\":{}}", err <nil>14431I1018 20:56:24.189142 1471497 cni.go:265] [cluster-density-0/cluster-density-5-d785dfd8-qmwfm 198b41e82453d92c2b7adb82f7421a39dd124df23867a4091011077b26cc4fe2] DEL starting CNI request [cluster-density-0/cluster-density-5-d785dfd8-qmwfm 198b41e82453d92c2b7adb82f7421a39dd124df23867a4091011077b26cc4fe2]14432W1018 20:56:24.196037 1471497 helper_linux.go:649] Failed to delete pod "cluster-density-0/cluster-density-5-d785dfd8-qmwfm" interface 198b41e82453d92: failed to lookup link 198b41e82453d92: Link not found14433I1018 20:56:24.206711 1471497 cni.go:286] [cluster-density-0/cluster-density-5-d785dfd8-qmwfm 198b41e82453d92c2b7adb82f7421a39dd124df23867a4091011077b26cc4fe2] DEL finished CNI request [cluster-density-0/cluster-density-5-d785dfd8-qmwfm 198b41e82453d92c2b7adb82f7421a39dd124df23867a4091011077b26cc4fe2], result "{\"dns\":{}}", err <nil>14434I1018 20:56:32.720852 1471497 cni.go:265] [open-cluster-management-agent-addon/klusterlet-addon-search-67d85457cd-cmtk2 2831d3f0aef872249aa5500ff6e131b59894592c904bbc7c80008cc118d928aa] ADD starting CNI request [open-cluster-management-agent-addon/klusterlet-addon-search-67d85457cd-cmtk2 2831d3f0aef872249aa5500ff6e131b59894592c904bbc7c80008cc118d928aa]14435I1018 20:56:32.724666 1471497 helper_linux.go:363] ConfigureOVS: namespace: open-cluster-management-agent-addon, podName: klusterlet-addon-search-67d85457cd-cmtk2, network: default, NAD default, SandboxID: "2831d3f0aef872249aa5500ff6e131b59894592c904bbc7c80008cc118d928aa", UID: "949bd994-e090-4592-b2e8-90d7ffd7c3f6", MAC: 0a:58:0a:84:00:22, IPs: [10.132.0.34/23]14436W1018 20:56:39.725285 1471497 helper_linux.go:432] [test/example 397422b49cdc906c512859a0db8a0ab244754d8d4f10d6cb76b1cf6587321eb9] pod uid 108d2d39-7757-4fb1-b6de-8408bb1c481d: timed out waiting for OVS port binding (ovn-installed) for 0a:58:0a:84:00:d9 [10.132.0.217/23]14437I1018 20:56:39.755237 1471497 cni.go:286] [test/example 397422b49cdc906c512859a0db8a0ab244754d8d4f10d6cb76b1cf6587321eb9] ADD finished CNI request [test/example 397422b49cdc906c512859a0db8a0ab244754d8d4f10d6cb76b1cf6587321eb9], result "", err failed to configure pod interface: timed out waiting for OVS port binding (ovn-installed) for 0a:58:0a:84:00:d9 [10.132.0.217/23]14438I1018 20:56:39.872892 1471497 cni.go:265] [test/example 397422b49cdc906c512859a0db8a0ab244754d8d4f10d6cb76b1cf6587321eb9] DEL starting CNI request [test/example 397422b49cdc906c512859a0db8a0ab244754d8d4f10d6cb76b1cf6587321eb9]14439W1018 20:56:39.879802 1471497 helper_linux.go:649] Failed to delete pod "test/example" interface 397422b49cdc906: failed to lookup link 397422b49cdc906: Link not found
      
      

            bbennett@redhat.com Ben Bennett
            feven@redhat.com Florian Even (Inactive)
            Anurag Saxena Anurag Saxena
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: