Loading...

Type: Bug
Resolution: Done
Priority: Major
Fix Version/s: None
Affects Version/s: FDP-24.H
Component/s: openvswitch3.3, openvswitch3.4
Labels:
None

Blocked:
False
Blocked Reason:

Hide

None

Show
None
Ready:
False
Pool Team:

rhel-sst-network-fastdatapath
Intelligence Requested:
Market:
Sub-System Group:

ssg_networking

SFDC Cases Links:
SFDC Cases Counter:
SFDC Cases Open:

Description:

add ovs bond port failed if the slave interface is created by i40e's vf. This issue was introduce by openvswitch3.3-3.3.0-56.el9fdp/openvswitch3.4-3.4.0-12.el9fdp

Version-Release number of selected component (if applicable):

kernel version: 5.14.0-427.42.1.el9_4.x86_64

openvswitch verion: openvswitch3.3-3.3.0-56.el9fdp/openvswitch3.4-3.4.0-12.el9fdp

How reproducible: 100%

Steps to Reproduce:

run below script

#!/bin/bash
systemctl stop openvswitch &>/dev/null
rm -rf /etc/openvswitch/*.db
rm -rf /var/lib/openvswitch/*
rm -rf /dev/hugepages/rtemap_*
echo 0 > /sys/class/net/ens1f0/device/sriov_numvfs
echo 0 > /sys/class/net/ens1f1/device/sriov_numvfs
rmmod irdma
rmmod i40e
systemctl restart openvswitch
modprobe i40e
ip link set ens1f0 up
ip link set ens1f1 up
echo 1 > /sys/class/net/ens1f0/device/sriov_numvfs
echo 1 > /sys/class/net/ens1f1/device/sriov_numvfs
ip link show ens1f0
ip link show ens1f1ip link set ens1f0 vf 0 spoofchk off trust on
ip link set ens1f1 vf 0 spoofchk off trust on
ovs-vsctl set Open_vSwitch . other_config={}
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem='8192,8192'
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-lcore-mask=0x100000000000000
ovs-vsctl --no-wait set Open_vSwitch . other_config:pmd-cpu-mask=0x1400000014000000
ovs-vsctl --no-wait set Open_vSwitch . other_config:vhost-iommu-support=true
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=truedriverctl -v set-override 0000:3b:02.0 vfio-pci
driverctl -v set-override 0000:3b:0a.0 vfio-pci
ovs-vsctl --may-exist add-br bondbridge -- set bridge bondbridge datapath_type=netdev
ovs-vsctl add-bond bondbridge active-backup ens1f0v0 ens1f1v0 lacp=off bond_mode=active-backup  \
-- set Interface ens1f0v0 type=dpdk options:dpdk-devargs=0000:3b:02.0 options:n_rxq=2 mtu_request=9000  \
-- set Interface ens1f1v0 type=dpdk options:dpdk-devargs=0000:3b:0a.0 options:n_rxq=2 mtu_request=9000
ovs-vsctl list port active-backup
ovs-vsctl show

Key error message

2024-11-13T07:25:36.037Z|00133|dpdk|ERR|iavf_dev_reset(): fail to send cmd VIRTCHNL_OP_RESET_VF
2024-11-13T07:25:36.037Z|00134|dpdk|ERR|iavf_queues_req_reset(): vf reset failed
2024-11-13T07:25:36.037Z|00135|dpdk|ERR|Port4 dev_configure = -32
2024-11-13T07:25:36.037Z|00136|netdev_dpdk|WARN|Interface ens1f0v0 eth_dev setup error Broken pipe
2024-11-13T07:25:36.037Z|00137|netdev_dpdk|ERR|Interface ens1f0v0(rxq:2 txq:5 lsc interrupt mode:false) configure error: Broken pipe
2024-11-13T07:25:36.037Z|00138|netdev_dpdk|INFO|ens1f0v0: rx-steering: default rss
2024-11-13T07:25:36.037Z|00139|dpif_netdev|ERR|Failed to set interface ens1f0v0 new configuration
2024-11-13T07:25:36.037Z|00140|dpif|WARN|netdev@ovs-netdev: failed to add ens1f0v0 as port: Invalid argument
2024-11-13T07:25:36.037Z|00141|bridge|WARN|could not add network device ens1f0v0 to ofproto (Invalid argument)

Full of openvswitch log

2024-11-13T07:25:27.270Z|00001|vlog|INFO|opened log file /var/log/openvswitch/ovs-vswitchd.log
2024-11-13T07:25:27.337Z|00002|ovs_numa|INFO|Discovered 32 CPU cores on NUMA node 1
2024-11-13T07:25:27.337Z|00003|ovs_numa|INFO|Discovered 32 CPU cores on NUMA node 0
2024-11-13T07:25:27.337Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes and 64 CPU cores
2024-11-13T07:25:27.337Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connecting...
2024-11-13T07:25:27.337Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connected
2024-11-13T07:25:27.338Z|00007|dpdk|INFO|DPDK Disabled - Use other_config:dpdk-init to enable
2024-11-13T07:25:27.342Z|00008|bridge|INFO|ovs-vswitchd (Open vSwitch) 3.3.3-56.el9fdp
2024-11-13T07:25:29.292Z|00009|dpdk|INFO|Using DPDK 23.11.2
2024-11-13T07:25:29.292Z|00010|dpdk|INFO|DPDK Enabled - initializing...
2024-11-13T07:25:29.292Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd -c 0x100000000000000 --socket-mem 8192,8192 --in-memory.
2024-11-13T07:25:29.296Z|00012|dpdk|INFO|EAL: Detected CPU lcores: 64
2024-11-13T07:25:29.296Z|00013|dpdk|INFO|EAL: Detected NUMA nodes: 2
2024-11-13T07:25:29.296Z|00014|dpdk|INFO|EAL: Detected static linkage of DPDK
2024-11-13T07:25:29.316Z|00015|dpdk|INFO|EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Permission denied
2024-11-13T07:25:29.316Z|00016|dpdk|INFO|EAL: Selected IOVA mode 'VA'
2024-11-13T07:25:29.316Z|00017|dpdk|WARN|EAL: No free 2048 kB hugepages reported on node 0
2024-11-13T07:25:29.316Z|00018|dpdk|WARN|EAL: No free 2048 kB hugepages reported on node 1
2024-11-13T07:25:29.317Z|00019|dpdk|INFO|EAL: VFIO support initialized
2024-11-13T07:25:33.467Z|00020|dpdk|INFO|EAL: Probe PCI driver: mlx5_pci (15b3:101f) device: 0000:5e:00.0 (socket 0)
2024-11-13T07:25:33.629Z|00021|dpdk|INFO|EAL: Probe PCI driver: mlx5_pci (15b3:101f) device: 0000:5e:00.1 (socket 0)
2024-11-13T07:25:33.794Z|00022|dpdk|INFO|EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:86:00.0 (socket 1)
2024-11-13T07:25:34.015Z|00023|dpdk|INFO|EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:86:00.1 (socket 1)
2024-11-13T07:25:34.233Z|00024|dpdk|WARN|TELEMETRY: No legacy callbacks, legacy socket not created
2024-11-13T07:25:34.233Z|00025|netdev_dpdk|INFO|Per port memory for DPDK devices disabled.
2024-11-13T07:25:34.234Z|00026|netdev_dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch
2024-11-13T07:25:34.234Z|00027|netdev_dpdk|INFO|IOMMU support for vhost-user-client enabled.
2024-11-13T07:25:34.234Z|00028|netdev_dpdk|INFO|POSTCOPY support for vhost-user-client disabled.
2024-11-13T07:25:34.235Z|00029|dpdk|INFO|DPDK Enabled - initialized
2024-11-13T07:25:34.236Z|00030|timeval|WARN|Unreasonably long 4945ms poll interval (14ms user, 4094ms system)
2024-11-13T07:25:34.236Z|00031|timeval|WARN|faults: 21099 minor, 0 major
2024-11-13T07:25:34.236Z|00032|timeval|WARN|context switches: 2619 voluntary, 71 involuntary
2024-11-13T07:25:34.236Z|00033|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour,  hash=80bb85b0:
2024-11-13T07:25:34.236Z|00034|coverage|INFO|nln_changed                3.6/sec     0.300/sec        0.0050/sec   total: 18
2024-11-13T07:25:34.236Z|00035|coverage|INFO|route_table_dump           2.2/sec     0.183/sec        0.0031/sec   total: 11
2024-11-13T07:25:34.236Z|00036|coverage|INFO|netlink_sent             117.2/sec     9.767/sec        0.1628/sec   total: 586
2024-11-13T07:25:34.236Z|00037|coverage|INFO|netlink_recv_jumbo        11.4/sec     0.950/sec        0.0158/sec   total: 57
2024-11-13T07:25:34.236Z|00038|coverage|INFO|netlink_received         133.4/sec    11.117/sec        0.1853/sec   total: 667
2024-11-13T07:25:34.236Z|00039|coverage|INFO|util_xalloc              3124.6/sec   260.383/sec        4.3397/sec   total: 15623
2024-11-13T07:25:34.236Z|00040|coverage|INFO|stream_open                0.2/sec     0.017/sec        0.0003/sec   total: 1
2024-11-13T07:25:34.236Z|00041|coverage|INFO|pstream_open               0.2/sec     0.017/sec        0.0003/sec   total: 1
2024-11-13T07:25:34.236Z|00042|coverage|INFO|seq_change               163.0/sec    13.583/sec        0.2264/sec   total: 815
2024-11-13T07:25:34.236Z|00043|coverage|INFO|poll_create_node         119.8/sec     9.983/sec        0.1664/sec   total: 599
2024-11-13T07:25:34.236Z|00044|coverage|INFO|txn_success                0.4/sec     0.033/sec        0.0006/sec   total: 2
2024-11-13T07:25:34.236Z|00045|coverage|INFO|txn_incomplete             1.4/sec     0.117/sec        0.0019/sec   total: 7
2024-11-13T07:25:34.236Z|00046|coverage|INFO|txn_unchanged              5.4/sec     0.450/sec        0.0075/sec   total: 27
2024-11-13T07:25:34.236Z|00047|coverage|INFO|hmap_expand              153.0/sec    12.750/sec        0.2125/sec   total: 765
2024-11-13T07:25:34.236Z|00048|coverage|INFO|hmap_pathological          1.2/sec     0.100/sec        0.0017/sec   total: 6
2024-11-13T07:25:34.236Z|00049|coverage|INFO|miniflow_malloc           44.4/sec     3.700/sec        0.0617/sec   total: 222
2024-11-13T07:25:34.236Z|00050|coverage|INFO|cmap_shrink               14.0/sec     1.167/sec        0.0194/sec   total: 70
2024-11-13T07:25:34.236Z|00051|coverage|INFO|cmap_expand               16.2/sec     1.350/sec        0.0225/sec   total: 81
2024-11-13T07:25:34.236Z|00052|coverage|INFO|bridge_reconfigure         2.8/sec     0.233/sec        0.0039/sec   total: 14
2024-11-13T07:25:34.236Z|00053|coverage|INFO|149 events never hit
2024-11-13T07:25:34.237Z|00054|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157 (59% CPU usage)
2024-11-13T07:25:34.237Z|00055|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (NETLINK_ROUTE<->NETLINK_ROUTE) at ../lib/netlink-socket.c:1418 (59% CPU usage)
2024-11-13T07:25:34.237Z|00056|pmd_perf|INFO|DPDK provided TSC frequency: 2890000 KHz
2024-11-13T07:25:34.238Z|00057|dpif_netdev_extract|INFO|Default MFEX Extract implementation is scalar.
2024-11-13T07:25:34.241Z|00058|dpif_netdev_impl|INFO|Default DPIF implementation is dpif_scalar.
2024-11-13T07:25:34.247Z|00059|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports recirculation
2024-11-13T07:25:34.247Z|00060|ofproto_dpif|INFO|netdev@ovs-netdev: VLAN header stack length probed as 1
2024-11-13T07:25:34.247Z|00061|ofproto_dpif|INFO|netdev@ovs-netdev: MPLS label stack length probed as 3
2024-11-13T07:25:34.247Z|00062|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports truncate action
2024-11-13T07:25:34.247Z|00063|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports unique flow ids
2024-11-13T07:25:34.247Z|00064|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports clone action
2024-11-13T07:25:34.247Z|00065|ofproto_dpif|INFO|netdev@ovs-netdev: Max sample nesting level probed as 10
2024-11-13T07:25:34.247Z|00066|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports eventmask in conntrack action
2024-11-13T07:25:34.247Z|00067|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_clear action
2024-11-13T07:25:34.247Z|00068|ofproto_dpif|INFO|netdev@ovs-netdev: Max dp_hash algorithm probed to be 1
2024-11-13T07:25:34.247Z|00069|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports check_pkt_len action
2024-11-13T07:25:34.247Z|00070|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports timeout policy in conntrack action
2024-11-13T07:25:34.247Z|00071|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_zero_snat
2024-11-13T07:25:34.247Z|00072|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports add_mpls action
2024-11-13T07:25:34.247Z|00073|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_state
2024-11-13T07:25:34.247Z|00074|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_zone
2024-11-13T07:25:34.247Z|00075|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_mark
2024-11-13T07:25:34.247Z|00076|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_label
2024-11-13T07:25:34.247Z|00077|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_state_nat
2024-11-13T07:25:34.247Z|00078|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_orig_tuple
2024-11-13T07:25:34.247Z|00079|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_orig_tuple6
2024-11-13T07:25:34.247Z|00080|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports IPv6 ND Extensions
2024-11-13T07:25:34.247Z|00081|ofproto_dpif_upcall|INFO|Setting n-handler-threads to 1, setting n-revalidator-threads to 1
2024-11-13T07:25:34.247Z|00082|ofproto_dpif_upcall|INFO|Starting 2 threads
2024-11-13T07:25:34.250Z|00083|dpif_netdev|INFO|pmd-rxq-affinity isolates PMD core
2024-11-13T07:25:34.250Z|00084|dpif_netdev|INFO|PMD auto load balance interval set to 1 mins
2024-11-13T07:25:34.250Z|00085|dpif_netdev|INFO|PMD auto load balance improvement threshold set to 25%
2024-11-13T07:25:34.250Z|00086|dpif_netdev|INFO|PMD auto load balance load threshold set to 95%
2024-11-13T07:25:34.250Z|00087|dpif_netdev|INFO|PMD auto load balance is disabled.
2024-11-13T07:25:34.250Z|00088|dpif_netdev|INFO|Default PMD thread max sleep:    0 us.
2024-11-13T07:25:34.250Z|00089|bridge|INFO|bridge bondbridge: added interface bondbridge on port 65534
2024-11-13T07:25:34.250Z|00090|bridge|INFO|bridge bondbridge: using datapath ID 00004ae80c70584a
2024-11-13T07:25:34.250Z|00091|connmgr|INFO|bondbridge: added service controller "punix:/var/run/openvswitch/bondbridge.mgmt"
2024-11-13T07:25:34.252Z|00092|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (NETLINK_ROUTE<->NETLINK_ROUTE) at ../lib/netlink-socket.c:1418 (59% CPU usage)
2024-11-13T07:25:34.252Z|00093|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157 (59% CPU usage)
2024-11-13T07:25:34.253Z|00094|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157 (59% CPU usage)
2024-11-13T07:25:34.253Z|00095|poll_loop|INFO|wakeup due to 0-ms timeout at ../vswitchd/bridge.c:3266 (59% CPU usage)
2024-11-13T07:25:34.253Z|00096|poll_loop|INFO|wakeup due to [POLLIN] on fd 22 (FIFO pipe:[220183]) at ../vswitchd/bridge.c:424 (59% CPU usage)
2024-11-13T07:25:34.258Z|00097|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157 (59% CPU usage)
2024-11-13T07:25:34.276Z|00098|dpdk|INFO|EAL: Using IOMMU type 1 (Type 1)
2024-11-13T07:25:34.554Z|00099|dpdk|INFO|EAL: Probe PCI driver: net_iavf (8086:154c) device: 0000:3b:0a.0 (socket 0)
2024-11-13T07:25:34.658Z|00100|netdev_dpdk|INFO|Device '0000:3b:0a.0' attached to DPDK
2024-11-13T07:25:34.661Z|00101|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 60, max sleep:    0 us.
2024-11-13T07:25:34.662Z|00102|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 60 created.
2024-11-13T07:25:34.664Z|00103|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 58, max sleep:    0 us.
2024-11-13T07:25:34.665Z|00104|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 58 created.
2024-11-13T07:25:34.667Z|00105|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 26, max sleep:    0 us.
2024-11-13T07:25:34.668Z|00106|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 26 created.
2024-11-13T07:25:34.671Z|00107|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 28, max sleep:    0 us.
2024-11-13T07:25:34.671Z|00108|dpif_netdev|INFO|PMD thread on numa_id: 0, core id: 28 created.
2024-11-13T07:25:34.671Z|00109|dpif_netdev|INFO|There are 4 pmd threads on numa node 0
2024-11-13T07:25:34.671Z|00110|dpdk|INFO|Device with port_id=4 already stopped
2024-11-13T07:25:34.677Z|00001|dpdk(pmd-c28/id:8)|INFO|PMD thread uses DPDK lcore 0.
2024-11-13T07:25:34.677Z|00001|dpdk(pmd-c26/id:9)|INFO|PMD thread uses DPDK lcore 1.
2024-11-13T07:25:34.686Z|00001|dpdk(pmd-c58/id:10)|INFO|PMD thread uses DPDK lcore 2.
2024-11-13T07:25:34.686Z|00001|dpdk(pmd-c60/id:11)|INFO|PMD thread uses DPDK lcore 3.
2024-11-13T07:25:34.746Z|00111|netdev_dpdk|INFO|Virtual function detected, HW_CRC_STRIP will be enabled
2024-11-13T07:25:34.746Z|00112|netdev_dpdk|INFO|ens1f1v0: disabled Tx outer udp checksum offloads for a net/ice, net/i40e, net/iavf or net/txgbe port.
2024-11-13T07:25:34.797Z|00113|dpdk|ERR|iavf_dev_reset(): fail to send cmd VIRTCHNL_OP_RESET_VF
2024-11-13T07:25:34.797Z|00114|dpdk|ERR|iavf_queues_req_reset(): vf reset failed
2024-11-13T07:25:34.797Z|00115|dpdk|ERR|Port4 dev_configure = -32
2024-11-13T07:25:34.797Z|00116|netdev_dpdk|WARN|Interface ens1f1v0 eth_dev setup error Broken pipe
2024-11-13T07:25:34.797Z|00117|netdev_dpdk|ERR|Interface ens1f1v0(rxq:2 txq:5 lsc interrupt mode:false) configure error: Broken pipe
2024-11-13T07:25:34.797Z|00118|netdev_dpdk|INFO|ens1f1v0: rx-steering: default rss
2024-11-13T07:25:34.797Z|00119|dpif_netdev|ERR|Failed to set interface ens1f1v0 new configuration
2024-11-13T07:25:34.797Z|00120|dpif|WARN|netdev@ovs-netdev: failed to add ens1f1v0 as port: Invalid argument
2024-11-13T07:25:34.797Z|00121|bridge|WARN|could not add network device ens1f1v0 to ofproto (Invalid argument)
2024-11-13T07:25:34.797Z|00122|dpdk|INFO|Device with port_id=4 already stopped
2024-11-13T07:25:34.797Z|00123|dpdk|INFO|EAL: Releasing PCI mapped resource for 0000:3b:0a.0
2024-11-13T07:25:34.797Z|00124|dpdk|INFO|EAL: Calling pci_unmap_resource for 0000:3b:0a.0 at 0x4201000000
2024-11-13T07:25:34.797Z|00125|dpdk|INFO|EAL: Calling pci_unmap_resource for 0000:3b:0a.0 at 0x4201010000
2024-11-13T07:25:35.032Z|00126|netdev_dpdk|INFO|Device '0000:3b:0a.0' has been removed and detached
2024-11-13T07:25:35.050Z|00127|dpdk|INFO|EAL: Using IOMMU type 1 (Type 1)
2024-11-13T07:25:35.330Z|00128|dpdk|INFO|EAL: Probe PCI driver: net_iavf (8086:154c) device: 0000:3b:02.0 (socket 0)
2024-11-13T07:25:35.434Z|00129|netdev_dpdk|INFO|Device '0000:3b:02.0' attached to DPDK
2024-11-13T07:25:35.434Z|00130|dpdk|INFO|Device with port_id=4 already stopped
2024-11-13T07:25:35.987Z|00131|netdev_dpdk|INFO|Virtual function detected, HW_CRC_STRIP will be enabled
2024-11-13T07:25:35.987Z|00132|netdev_dpdk|INFO|ens1f0v0: disabled Tx outer udp checksum offloads for a net/ice, net/i40e, net/iavf or net/txgbe port.
2024-11-13T07:25:36.037Z|00133|dpdk|ERR|iavf_dev_reset(): fail to send cmd VIRTCHNL_OP_RESET_VF
2024-11-13T07:25:36.037Z|00134|dpdk|ERR|iavf_queues_req_reset(): vf reset failed
2024-11-13T07:25:36.037Z|00135|dpdk|ERR|Port4 dev_configure = -32
2024-11-13T07:25:36.037Z|00136|netdev_dpdk|WARN|Interface ens1f0v0 eth_dev setup error Broken pipe
2024-11-13T07:25:36.037Z|00137|netdev_dpdk|ERR|Interface ens1f0v0(rxq:2 txq:5 lsc interrupt mode:false) configure error: Broken pipe
2024-11-13T07:25:36.037Z|00138|netdev_dpdk|INFO|ens1f0v0: rx-steering: default rss
2024-11-13T07:25:36.037Z|00139|dpif_netdev|ERR|Failed to set interface ens1f0v0 new configuration
2024-11-13T07:25:36.037Z|00140|dpif|WARN|netdev@ovs-netdev: failed to add ens1f0v0 as port: Invalid argument
2024-11-13T07:25:36.037Z|00141|bridge|WARN|could not add network device ens1f0v0 to ofproto (Invalid argument)
2024-11-13T07:25:36.037Z|00142|dpdk|INFO|Device with port_id=4 already stopped
2024-11-13T07:25:36.038Z|00143|dpdk|INFO|EAL: Releasing PCI mapped resource for 0000:3b:02.0
2024-11-13T07:25:36.038Z|00144|dpdk|INFO|EAL: Calling pci_unmap_resource for 0000:3b:02.0 at 0x4201014000
2024-11-13T07:25:36.038Z|00145|dpdk|INFO|EAL: Calling pci_unmap_resource for 0000:3b:02.0 at 0x4201024000
2024-11-13T07:25:36.280Z|00146|netdev_dpdk|INFO|Device '0000:3b:02.0' has been removed and detached
2024-11-13T07:25:36.280Z|00147|timeval|WARN|Unreasonably long 2022ms poll interval (85ms user, 451ms system)
2024-11-13T07:25:36.280Z|00148|timeval|WARN|faults: 11558 minor, 0 major
2024-11-13T07:25:36.280Z|00149|timeval|WARN|context switches: 5351 voluntary, 126 involuntary
2024-11-13T07:25:36.280Z|00150|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour,  hash=f73e61e6:
2024-11-13T07:25:36.281Z|00151|coverage|INFO|nln_changed                3.6/sec     0.300/sec        0.0050/sec   total: 22
2024-11-13T07:25:36.281Z|00152|coverage|INFO|route_table_dump           2.2/sec     0.183/sec        0.0031/sec   total: 14
2024-11-13T07:25:36.281Z|00153|coverage|INFO|netlink_sent             117.2/sec     9.767/sec        0.1628/sec   total: 787
2024-11-13T07:25:36.281Z|00154|coverage|INFO|netlink_recv_jumbo        11.4/sec     0.950/sec        0.0158/sec   total: 95
2024-11-13T07:25:36.281Z|00155|coverage|INFO|netlink_received         133.4/sec    11.117/sec        0.1853/sec   total: 885
2024-11-13T07:25:36.281Z|00156|coverage|INFO|netdev_get_ethtool         0.0/sec     0.000/sec        0.0000/sec   total: 3
2024-11-13T07:25:36.281Z|00157|coverage|INFO|netdev_set_hwaddr          0.0/sec     0.000/sec        0.0000/sec   total: 1
2024-11-13T07:25:36.281Z|00158|coverage|INFO|netdev_set_policing        0.0/sec     0.000/sec        0.0000/sec   total: 2
2024-11-13T07:25:36.281Z|00159|coverage|INFO|util_xalloc              3124.6/sec   260.383/sec        4.3397/sec   total: 21143
2024-11-13T07:25:36.281Z|00160|coverage|INFO|long_poll_interval         0.0/sec     0.000/sec        0.0000/sec   total: 1
2024-11-13T07:25:36.281Z|00161|coverage|INFO|stream_open                0.2/sec     0.017/sec        0.0003/sec   total: 1
2024-11-13T07:25:36.281Z|00162|coverage|INFO|pstream_open               0.2/sec     0.017/sec        0.0003/sec   total: 3
2024-11-13T07:25:36.281Z|00163|coverage|INFO|seq_change               163.0/sec    13.583/sec        0.2264/sec   total: 3719
2024-11-13T07:25:36.281Z|00164|coverage|INFO|poll_zero_timeout          0.0/sec     0.000/sec        0.0000/sec   total: 3
2024-11-13T07:25:36.281Z|00165|coverage|INFO|poll_create_node         119.8/sec     9.983/sec        0.1664/sec   total: 835
2024-11-13T07:25:36.281Z|00166|coverage|INFO|txn_success                0.4/sec     0.033/sec        0.0006/sec   total: 4
2024-11-13T07:25:36.281Z|00167|coverage|INFO|txn_incomplete             1.4/sec     0.117/sec        0.0019/sec   total: 10
2024-11-13T07:25:36.281Z|00168|coverage|INFO|txn_unchanged              5.4/sec     0.450/sec        0.0075/sec   total: 30
2024-11-13T07:25:36.281Z|00169|coverage|INFO|netdev_get_stats           0.0/sec     0.000/sec        0.0000/sec   total: 1
2024-11-13T07:25:36.281Z|00170|coverage|INFO|hmap_shrink                0.0/sec     0.000/sec        0.0000/sec   total: 1
2024-11-13T07:25:36.281Z|00171|coverage|INFO|hmap_expand              153.0/sec    12.750/sec        0.2125/sec   total: 920
2024-11-13T07:25:36.281Z|00172|coverage|INFO|hmap_pathological          1.2/sec     0.100/sec        0.0017/sec   total: 6
2024-11-13T07:25:36.281Z|00173|coverage|INFO|miniflow_malloc           44.4/sec     3.700/sec        0.0617/sec   total: 303
2024-11-13T07:25:36.281Z|00174|coverage|INFO|flow_extract               0.0/sec     0.000/sec        0.0000/sec   total: 4
2024-11-13T07:25:36.281Z|00175|coverage|INFO|dpif_port_add              0.0/sec     0.000/sec        0.0000/sec   total: 3
2024-11-13T07:25:36.281Z|00176|coverage|INFO|dpif_flow_put_error        0.0/sec     0.000/sec        0.0000/sec   total: 1
2024-11-13T07:25:36.281Z|00177|coverage|INFO|dpif_flow_put              0.0/sec     0.000/sec        0.0000/sec   total: 25
2024-11-13T07:25:36.281Z|00178|coverage|INFO|dpif_flow_get              0.0/sec     0.000/sec        0.0000/sec   total: 24
2024-11-13T07:25:36.281Z|00179|coverage|INFO|dpif_flow_flush            0.0/sec     0.000/sec        0.0000/sec   total: 1
2024-11-13T07:25:36.281Z|00180|coverage|INFO|dpif_flow_del              0.0/sec     0.000/sec        0.0000/sec   total: 24
2024-11-13T07:25:36.281Z|00181|coverage|INFO|dpif_execute               0.0/sec     0.000/sec        0.0000/sec   total: 6
2024-11-13T07:25:36.281Z|00182|coverage|INFO|cmap_shrink               14.0/sec     1.167/sec        0.0194/sec   total: 117
2024-11-13T07:25:36.281Z|00183|coverage|INFO|cmap_expand               16.2/sec     1.350/sec        0.0225/sec   total: 137
2024-11-13T07:25:36.281Z|00184|coverage|INFO|ccmap_shrink               0.0/sec     0.000/sec        0.0000/sec   total: 24
2024-11-13T07:25:36.281Z|00185|coverage|INFO|rev_flow_table             0.0/sec     0.000/sec        0.0000/sec   total: 1
2024-11-13T07:25:36.281Z|00186|coverage|INFO|rev_reconfigure            0.0/sec     0.000/sec        0.0000/sec   total: 2
2024-11-13T07:25:36.281Z|00187|coverage|INFO|ofproto_update_port        0.0/sec     0.000/sec        0.0000/sec   total: 2
2024-11-13T07:25:36.281Z|00188|coverage|INFO|ofproto_flush              0.0/sec     0.000/sec        0.0000/sec   total: 1
2024-11-13T07:25:36.281Z|00189|coverage|INFO|bridge_reconfigure         2.8/sec     0.233/sec        0.0039/sec   total: 17
2024-11-13T07:25:36.281Z|00190|coverage|INFO|129 events never hit
2024-11-13T07:25:36.281Z|00191|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157 (59% CPU usage)
2024-11-13T07:25:36.299Z|00192|dpdk|INFO|EAL: Using IOMMU type 1 (Type 1)
2024-11-13T07:25:36.578Z|00193|dpdk|INFO|EAL: Probe PCI driver: net_iavf (8086:154c) device: 0000:3b:0a.0 (socket 0)
2024-11-13T07:25:36.682Z|00194|netdev_dpdk|INFO|Device '0000:3b:0a.0' attached to DPDK
2024-11-13T07:25:36.682Z|00195|dpdk|INFO|Device with port_id=4 already stopped
2024-11-13T07:25:37.235Z|00196|netdev_dpdk|INFO|Virtual function detected, HW_CRC_STRIP will be enabled
2024-11-13T07:25:37.235Z|00197|netdev_dpdk|INFO|ens1f1v0: disabled Tx outer udp checksum offloads for a net/ice, net/i40e, net/iavf or net/txgbe port.
2024-11-13T07:25:37.485Z|00198|dpdk|INFO|iavf_set_rx_function(): request RXDID[1] in Queue[0] is legacy, set rx_pkt_burst as legacy for all queues
2024-11-13T07:25:37.486Z|00199|dpdk|INFO|iavf_set_rx_function(): request RXDID[1] in Queue[1] is legacy, set rx_pkt_burst as legacy for all queues
2024-11-13T07:25:38.285Z|00200|netdev_dpdk|INFO|Port 4: da:58:b1:49:a3:1c
2024-11-13T07:25:38.285Z|00201|netdev_dpdk|INFO|ens1f1v0: rx-steering: default rss
2024-11-13T07:25:38.285Z|00202|dpif_netdev|INFO|Performing pmd to rx queue assignment using cycles algorithm.
2024-11-13T07:25:38.285Z|00203|dpif_netdev|INFO|Core 26 on numa node 0 assigned port 'ens1f1v0' rx queue 0 (measured processing cycles 0).
2024-11-13T07:25:38.285Z|00204|dpif_netdev|INFO|Core 28 on numa node 0 assigned port 'ens1f1v0' rx queue 1 (measured processing cycles 0).
2024-11-13T07:25:38.285Z|00205|bridge|INFO|bridge bondbridge: added interface ens1f1v0 on port 1
2024-11-13T07:25:38.714Z|00206|dpdk|INFO|EAL: Probe PCI driver: net_iavf (8086:154c) device: 0000:3b:02.0 (socket 0)
2024-11-13T07:25:38.817Z|00207|netdev_dpdk|INFO|Device '0000:3b:02.0' attached to DPDK
2024-11-13T07:25:38.817Z|00208|dpdk|INFO|Device with port_id=5 already stopped
2024-11-13T07:25:38.817Z|00209|netdev_dpdk|INFO|Virtual function detected, HW_CRC_STRIP will be enabled
2024-11-13T07:25:38.817Z|00210|netdev_dpdk|INFO|ens1f0v0: disabled Tx outer udp checksum offloads for a net/ice, net/i40e, net/iavf or net/txgbe port.
2024-11-13T07:25:39.068Z|00211|dpdk|INFO|iavf_set_rx_function(): request RXDID[1] in Queue[0] is legacy, set rx_pkt_burst as legacy for all queues
2024-11-13T07:25:39.068Z|00212|dpdk|INFO|iavf_set_rx_function(): request RXDID[1] in Queue[1] is legacy, set rx_pkt_burst as legacy for all queues
2024-11-13T07:25:39.867Z|00213|netdev_dpdk|INFO|Port 5: ea:b6:44:c3:32:6b
2024-11-13T07:25:39.867Z|00214|netdev_dpdk|INFO|ens1f0v0: rx-steering: default rss
2024-11-13T07:25:39.867Z|00215|dpif_netdev|INFO|Performing pmd to rx queue assignment using cycles algorithm.
2024-11-13T07:25:39.867Z|00216|dpif_netdev|INFO|Core 26 on numa node 0 assigned port 'ens1f1v0' rx queue 0 (measured processing cycles 0).
2024-11-13T07:25:39.867Z|00217|dpif_netdev|INFO|Core 28 on numa node 0 assigned port 'ens1f1v0' rx queue 1 (measured processing cycles 0).
2024-11-13T07:25:39.867Z|00218|dpif_netdev|INFO|Core 58 on numa node 0 assigned port 'ens1f0v0' rx queue 0 (measured processing cycles 0).
2024-11-13T07:25:39.867Z|00219|dpif_netdev|INFO|Core 60 on numa node 0 assigned port 'ens1f0v0' rx queue 1 (measured processing cycles 0).
2024-11-13T07:25:39.867Z|00220|bridge|INFO|bridge bondbridge: added interface ens1f0v0 on port 2
2024-11-13T07:25:40.067Z|00221|bond|INFO|member ens1f1v0: enabled
2024-11-13T07:25:40.067Z|00222|bond|INFO|member ens1f0v0: enabled
2024-11-13T07:25:40.067Z|00223|timeval|WARN|Unreasonably long 3786ms poll interval (13ms user, 109ms system)
2024-11-13T07:25:40.067Z|00224|timeval|WARN|faults: 39 minor, 0 major
2024-11-13T07:25:40.067Z|00225|timeval|WARN|context switches: 7623 voluntary, 76 involuntary
2024-11-13T07:25:40.067Z|00226|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour,  hash=542657ec:
2024-11-13T07:25:40.067Z|00227|coverage|INFO|nln_changed                0.8/sec     0.367/sec        0.0061/sec   total: 22
2024-11-13T07:25:40.067Z|00228|coverage|INFO|route_table_dump           0.6/sec     0.233/sec        0.0039/sec   total: 14
2024-11-13T07:25:40.067Z|00229|coverage|INFO|netlink_sent              40.2/sec    13.117/sec        0.2186/sec   total: 787
2024-11-13T07:25:40.067Z|00230|coverage|INFO|netlink_recv_jumbo         7.6/sec     1.583/sec        0.0264/sec   total: 95
2024-11-13T07:25:40.067Z|00231|coverage|INFO|netlink_received          43.6/sec    14.750/sec        0.2458/sec   total: 885
2024-11-13T07:25:40.067Z|00232|coverage|INFO|netdev_get_ethtool         0.6/sec     0.050/sec        0.0008/sec   total: 3
2024-11-13T07:25:40.067Z|00233|coverage|INFO|netdev_set_hwaddr          0.2/sec     0.017/sec        0.0003/sec   total: 1
2024-11-13T07:25:40.067Z|00234|coverage|INFO|netdev_set_policing        0.4/sec     0.033/sec        0.0006/sec   total: 2
2024-11-13T07:25:40.067Z|00235|coverage|INFO|util_xalloc              1125.0/sec   354.133/sec        5.9022/sec   total: 22630
2024-11-13T07:25:40.067Z|00236|coverage|INFO|long_poll_interval         0.2/sec     0.017/sec        0.0003/sec   total: 2
2024-11-13T07:25:40.067Z|00237|coverage|INFO|stream_open                0.0/sec     0.017/sec        0.0003/sec   total: 1
2024-11-13T07:25:40.067Z|00238|coverage|INFO|pstream_open               0.4/sec     0.050/sec        0.0008/sec   total: 3
2024-11-13T07:25:40.068Z|00239|coverage|INFO|seq_change               1202.0/sec   113.750/sec        1.8958/sec   total: 42499
2024-11-13T07:25:40.068Z|00240|coverage|INFO|poll_zero_timeout          0.6/sec     0.050/sec        0.0008/sec   total: 3
2024-11-13T07:25:40.068Z|00241|coverage|INFO|poll_create_node          52.8/sec    14.383/sec        0.2397/sec   total: 869
2024-11-13T07:25:40.068Z|00242|coverage|INFO|txn_success                0.4/sec     0.067/sec        0.0011/sec   total: 4
2024-11-13T07:25:40.068Z|00243|coverage|INFO|txn_incomplete             0.6/sec     0.167/sec        0.0028/sec   total: 10
2024-11-13T07:25:40.068Z|00244|coverage|INFO|txn_unchanged              0.6/sec     0.500/sec        0.0083/sec   total: 30
2024-11-13T07:25:40.068Z|00245|coverage|INFO|netdev_get_stats           0.2/sec     0.017/sec        0.0003/sec   total: 3
2024-11-13T07:25:40.068Z|00246|coverage|INFO|hmap_shrink                0.2/sec     0.017/sec        0.0003/sec   total: 3
2024-11-13T07:25:40.068Z|00247|coverage|INFO|hmap_expand               32.0/sec    15.417/sec        0.2569/sec   total: 992
2024-11-13T07:25:40.068Z|00248|coverage|INFO|hmap_pathological          0.0/sec     0.100/sec        0.0017/sec   total: 6
2024-11-13T07:25:40.068Z|00249|coverage|INFO|miniflow_malloc           16.2/sec     5.050/sec        0.0842/sec   total: 303
2024-11-13T07:25:40.068Z|00250|coverage|INFO|flow_extract               0.8/sec     0.067/sec        0.0011/sec   total: 4
2024-11-13T07:25:40.068Z|00251|coverage|INFO|dpif_port_add              0.6/sec     0.050/sec        0.0008/sec   total: 5
2024-11-13T07:25:40.068Z|00252|coverage|INFO|dpif_flow_put_error        0.2/sec     0.017/sec        0.0003/sec   total: 1
2024-11-13T07:25:40.068Z|00253|coverage|INFO|dpif_flow_put              5.0/sec     0.417/sec        0.0069/sec   total: 25
2024-11-13T07:25:40.068Z|00254|coverage|INFO|dpif_flow_get              4.8/sec     0.400/sec        0.0067/sec   total: 24
2024-11-13T07:25:40.068Z|00255|coverage|INFO|dpif_flow_flush            0.2/sec     0.017/sec        0.0003/sec   total: 1
2024-11-13T07:25:40.068Z|00256|coverage|INFO|dpif_flow_del              4.8/sec     0.400/sec        0.0067/sec   total: 24
2024-11-13T07:25:40.068Z|00257|coverage|INFO|dpif_execute               1.2/sec     0.100/sec        0.0017/sec   total: 6
2024-11-13T07:25:40.068Z|00258|coverage|INFO|cmap_shrink                9.4/sec     1.950/sec        0.0325/sec   total: 117
2024-11-13T07:25:40.068Z|00259|coverage|INFO|cmap_expand               11.2/sec     2.283/sec        0.0381/sec   total: 137
2024-11-13T07:25:40.068Z|00260|coverage|INFO|ccmap_shrink               4.8/sec     0.400/sec        0.0067/sec   total: 24
2024-11-13T07:25:40.068Z|00261|coverage|INFO|rev_flow_table             0.2/sec     0.017/sec        0.0003/sec   total: 1
2024-11-13T07:25:40.068Z|00262|coverage|INFO|rev_reconfigure            0.4/sec     0.033/sec        0.0006/sec   total: 3
2024-11-13T07:25:40.068Z|00263|coverage|INFO|ofproto_update_port        0.4/sec     0.033/sec        0.0006/sec   total: 4
2024-11-13T07:25:40.068Z|00264|coverage|INFO|ofproto_flush              0.2/sec     0.017/sec        0.0003/sec   total: 1
2024-11-13T07:25:40.068Z|00265|coverage|INFO|bridge_reconfigure         0.6/sec     0.283/sec        0.0047/sec   total: 18
2024-11-13T07:25:40.068Z|00266|coverage|INFO|129 events never hit
2024-11-13T07:25:40.068Z|00267|bond|INFO|bond active-backup: active member is now ens1f0v0
2024-11-13T07:25:40.418Z|00268|memory|INFO|397860 kB peak resident set size after 13.1 seconds
2024-11-13T07:25:40.418Z|00269|memory|INFO|handlers:1 idl-cells-Open_vSwitch:188 idl-outstanding-txns-Open_vSwitch:2 ports:3 revalidators:1 rules:5
2024-11-13T07:25:41.040Z|00270|netdev_dpdk|INFO|Virtual function detected, HW_CRC_STRIP will be enabled
2024-11-13T07:25:41.040Z|00271|netdev_dpdk|INFO|ens1f1v0: disabled Tx outer udp checksum offloads for a net/ice, net/i40e, net/iavf or net/txgbe port.
2024-11-13T07:25:41.290Z|00272|dpdk|INFO|iavf_set_rx_function(): request RXDID[1] in Queue[0] is legacy, set rx_pkt_burst as legacy for all queues
2024-11-13T07:25:41.290Z|00273|dpdk|INFO|iavf_set_rx_function(): request RXDID[1] in Queue[1] is legacy, set rx_pkt_burst as legacy for all queues
2024-11-13T07:25:41.990Z|00274|netdev_dpdk|INFO|Port 4: 0e:13:7e:a8:0b:9e
2024-11-13T07:25:41.990Z|00275|netdev_dpdk|INFO|ens1f1v0: rx-steering: default rss
2024-11-13T07:25:41.990Z|00276|dpif_netdev|INFO|Performing pmd to rx queue assignment using cycles algorithm.
2024-11-13T07:25:41.990Z|00277|dpif_netdev|INFO|Core 26 on numa node 0 assigned port 'ens1f1v0' rx queue 0 (measured processing cycles 0).
2024-11-13T07:25:41.990Z|00278|dpif_netdev|INFO|Core 28 on numa node 0 assigned port 'ens1f1v0' rx queue 1 (measured processing cycles 0).
2024-11-13T07:25:41.990Z|00279|dpif_netdev|INFO|Core 58 on numa node 0 assigned port 'ens1f0v0' rx queue 0 (measured processing cycles 0).
2024-11-13T07:25:41.990Z|00280|dpif_netdev|INFO|Core 60 on numa node 0 assigned port 'ens1f0v0' rx queue 1 (measured processing cycles 0).
2024-11-13T07:25:41.990Z|00281|timeval|WARN|Unreasonably long 1572ms poll interval (6ms user, 5ms system)
2024-11-13T07:25:41.990Z|00282|timeval|WARN|context switches: 6016 voluntary, 26 involuntary

Details

Description

Attachments

Easy Agile Planning Poker

Activity

People

Dates