Loading...

Type: Bug
Resolution: Done
Priority: Normal
Fix Version/s: None
Affects Version/s: 4.10.z
Component/s: Networking / ovn-kubernetes
Labels:
None

Severity:
Moderate
Regression:
None
Sprint:
SDN Sprint 225, SDN Sprint 226
sprint_count:
2
Release Blocker:
Rejected
Blocked:
False
Blocked Reason:

Hide

None

Show
None
Target Version:

4.12.0

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

PX Impact Score:
PX Priority Data:

Cluster version 4.10.30

configure-ovs.sh fails when it activates clones connections if the original connection file targets a non-existing interface name.

E.g.:

                                                                                                                  
  $ egrep '^[0-9]' ./sos_commands/networking/ip_-s_-d_link                                                                   
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000                   
  2: enp36s0f3u1u6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 1000
  3: ens4f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000               
  4: ens4f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000           
  5: ens4f2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000           
  6: ens4f3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000           
  7: ens1f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000
  8: ens1f1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000
  9: ens3f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master bond1 state UP mode DEFAULT group default qlen 1000
  10: ens3f1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master bond1 state UP mode DEFAULT group default qlen 1000
  11: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000                  
  12: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000
  13: ovn-k8s-mp0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8900 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
  14: br-int: <BROADCAST,MULTICAST> mtu 8900 qdisc noop state DOWN mode DEFAULT group default qlen 1000                      
  16: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default qlen 1000
  17: bond1.3201@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default qlen 1000
  19: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default qlen 1000

When connections ens2f0 and ens2f1 are present (for whatever reason, but this was seen in an actual customer environment):

  $ cat sos_commands/networkmanager/nmcli_con                                                                             
  NAME              UUID                                  TYPE      DEVICE                                                
  bond0             ....................................  bond      bond0                                                 
  Wired Connection  ....................................  ethernet  enp36s0f3u1u6                                         
  bond1             ....................................  bond      bond1                                                 
  bond1.3201        ....................................  vlan      bond1.3201                                            
  ens1f0            ....................................  ethernet  ens1f0                                                
  ens1f1            ....................................  ethernet  ens1f1                                                
  ens3f0            ....................................  ethernet  ens3f0                                                
  ens3f1            ....................................  ethernet  ens3f1                                                
  ens2f0            ....................................  ethernet  --                                                    
  ens2f1            ....................................  ethernet  --                                                    
  ens6f0            ....................................  ethernet  --                                                    
  ens6f1            ....................................  ethernet  --

Then ovs-configure fails with:

Sep 19 10:39:19 localhost.localdomain configure-ovs.sh[20607]: + for i in {1..10}
Sep 19 10:39:19 localhost.localdomain configure-ovs.sh[20607]: + echo 'Attempt 1 to bring up connection ens2f0-slave-ovs-clone'
Sep 19 10:39:19 localhost.localdomain configure-ovs.sh[20607]: Attempt 1 to bring up connection ens2f0-slave-ovs-clone
Sep 19 10:39:19 localhost.localdomain configure-ovs.sh[20607]: + nmcli conn up ens2f0-slave-ovs-clone
Sep 19 10:39:19 localhost.localdomain configure-ovs.sh[20607]: Error: Connection activation failed: No suitable device found for this connection (device enp36s0f3u1u6 not available because profile is not compatible with device (mismatchi
ng interface name)).
Sep 19 10:39:19 localhost.localdomain configure-ovs.sh[20607]: + s=4
Sep 19 10:39:19 localhost.localdomain configure-ovs.sh[20607]: + sleep 5
Sep 19 10:39:22 localhost.localdomain configure-ovs.sh[20607]: + for i in {1..10}
Sep 19 10:39:22 localhost.localdomain configure-ovs.sh[20607]: + echo 'Attempt 2 to bring up connection ens2f0-slave-ovs-clone'
Sep 19 10:39:22 localhost.localdomain configure-ovs.sh[20607]: Attempt 2 to bring up connection ens2f0-slave-ovs-clone
Sep 19 10:39:22 localhost.localdomain configure-ovs.sh[20607]: + nmcli conn up ens2f0-slave-ovs-clone
Sep 19 10:39:22 localhost.localdomain configure-ovs.sh[20607]: Error: Connection activation failed: No suitable device found for this connection (device enp36s0f3u1u6 not available because profile is not compatible with device (mismatchi
ng interface name)).
(...)
Sep 19 10:40:03 localhost.localdomain configure-ovs.sh[20607]: + echo 'Attempt 10 to bring up connection ens2f0-slave-ovs-clone'
Sep 19 10:40:03 localhost.localdomain configure-ovs.sh[20607]: Attempt 10 to bring up connection ens2f0-slave-ovs-clone
Sep 19 10:40:03 localhost.localdomain configure-ovs.sh[20607]: + nmcli conn up ens2f0-slave-ovs-clone
Sep 19 10:40:03 localhost.localdomain configure-ovs.sh[20607]: Error: Connection activation failed: No suitable device found for this connection (device enp36s0f3u1u6 not available because profile is not compatible with device (mismatching interface name)).
Sep 19 10:40:03 localhost.localdomain configure-ovs.sh[20607]: + s=4
Sep 19 10:40:03 localhost.localdomain configure-ovs.sh[20607]: + sleep 5
Sep 19 10:40:08 localhost.localdomain configure-ovs.sh[20607]: + '[' 4 -eq 0 ']'
Sep 19 10:40:08 localhost.localdomain configure-ovs.sh[20607]: + echo 'ERROR: Cannot bring up connection ens2f0-slave-ovs-clone after 10 attempts'
Sep 19 10:40:08 localhost.localdomain configure-ovs.sh[20607]: ERROR: Cannot bring up connection ens2f0-slave-ovs-clone after 10 attempts
Sep 19 10:40:08 localhost.localdomain configure-ovs.sh[20607]: + return 4
Sep 19 10:40:08 localhost.localdomain configure-ovs.sh[20607]: + handle_exit
Sep 19 10:40:08 localhost.localdomain configure-ovs.sh[20607]: + e=4
Sep 19 10:40:08 localhost.localdomain configure-ovs.sh[20607]: + '[' 4 -eq 0 ']'
Sep 19 10:40:08 localhost.localdomain configure-ovs.sh[20607]: + echo 'ERROR: configure-ovs exited with error: 4'
Sep 19 10:40:08 localhost.localdomain configure-ovs.sh[20607]: ERROR: configure-ovs exited with error: 4

links to

openshift/machine-config-operator#3345: Jira OCPBUGS-1503: configure-ovs.sh: Do not fail bond with invalid slave device names

Details

Description

Attachments

Issue Links

Easy Agile Planning Poker

Activity

People

Dates