Description of problem:

When two VMs are having macsec interface to communicate to each other with the support of wpa_supplicant for keys exchange all works fine.

When 3-rd VM is added to the communication, keys are still invalidating and communication fails randomly.

Sep 07 13:36:11 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:11 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:13 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:13 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:15 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:15 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:17 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:17 H3 wpa_supplicant[48094]: KaY: The key server is not elected
Sep 07 13:36:17 H3 wpa_supplicant[48094]: KaY: Discarding Rx MKPDU: decode of parameter set type (4) failed
Sep 07 13:36:20 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:21 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:21 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:21 H3 wpa_supplicant[48094]: KaY: Old key is invalid
Sep 07 13:36:22 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:23 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:25 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU
Sep 07 13:36:27 H3 wpa_supplicant[48094]: KaY: duplicated SCI detected - maybe active attacker or peer selected new MI - ignore MKPDU

Version-Release number of selected component (if applicable):

1. nmcli -v
   nmcli tool, version 1.26.0-6.el8

1. NetworkManager -V
   1.26.0-6.el8

1. uname -r
   4.18.0-234.el8.x86_64

1. wpa_supplicant -v
   wpa_supplicant v2.9
   Copyright (c) 2003-2019, Jouni Malinen <j@w1.fi> and contributors

1. cat /etc/redhat-release
   Red Hat Enterprise Linux release 8.4 Beta (Ootpa)

How reproducible:

3 VMs having a NIC in a bridge br1 on the host.

let the bridge accept the macsec frames

1. echo 8 > /sys/class/net/br1/bridge/group_fwd_mask

VM1:

1. cexport MKA_CAK=$(dd if=/dev/urandom count=16 bs=1 2> /dev/null | hexdump -e '1/2 "%02x"')
2. cexport MKA_CKN=$(dd if=/dev/urandom count=32 bs=1 2> /dev/null | hexdump -e '1/2 "%02x"')

1. echo $MKA_CAK
   17a2b432c979691cdf9e3af6568171b8

1. echo $MKA_CKN
   c640a44ca5116c584b3f278066c3ca182c9cba061681f08ee5acf9cc1d518435

1. echo -n $MKA_CKN | wc -m
   64

1. echo -n $MKA_CAK | wc -m
   32

1. nmcli connection add type macsec con-name macsec_test ifname macsec0 macsec.parent ens11 macsec.mode psk macsec.mka-cak $MKA_CAK macsec.mka-ckn $MKA_CKN macsec.encrypt yes ipv4.method manual ipv4.addresses 10.10.10.1/24

1. nmcli connection up macsec_test

VM2:

1. export MKA_CAK=17a2b432c979691cdf9e3af6568171b8
2. export MKA_CKN=c640a44ca5116c584b3f278066c3ca182c9cba061681f08ee5acf9cc1d518435

1. nmcli connection add type macsec con-name macsec_test ifname macsec0 macsec.parent ens4 macsec.mode psk macsec.mka-cak $MKA_CAK macsec.mka-ckn $MKA_CKN macsec.encrypt yes ipv4.method manual ipv4.addresses 10.10.10.2/24

1. nmcli connection up macsec_test

Till here all works fine.

[root@H1 ~]# ip macsec show
24: macsec0: protect on validate strict sc off sa off encrypt on send_sci on end_station off scb off replay off
cipher suite: GCM-AES-128, using ICV length 16
TXSC: 52540074d8cd0001 on SA 0
0: PN 1, state on, key 0987862f26640c3167f0e2be01000000
RXSC: 5254000cca4f0001, state on
0: PN 4, state on, key 0987862f26640c3167f0e2be01000000

[root@H2 ~]# ip macsec show
11: macsec0: protect on validate strict sc off sa off encrypt on send_sci on end_station off scb off replay off
cipher suite: GCM-AES-128, using ICV length 16
TXSC: 5254000cca4f0001 on SA 0
0: PN 4, state on, key 0987862f26640c3167f0e2be01000000
RXSC: 52540074d8cd0001, state on
0: PN 1, state on, key 0987862f26640c3167f0e2be01000000

[root@H2 ~]# ping 10.10.10.1 -c 1
PING 10.10.10.1 (10.10.10.1) 56(84) bytes of data.
64 bytes from 10.10.10.1: icmp_seq=1 ttl=64 time=0.639 ms

— 10.10.10.1 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.639/0.639/0.639/0.000 ms

[root@H1 ~]# ping 10.10.10.2 -c1
PING 10.10.10.2 (10.10.10.2) 56(84) bytes of data.
64 bytes from 10.10.10.2: icmp_seq=1 ttl=64 time=0.369 ms

— 10.10.10.2 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.369/0.369/0.369/0.000 ms

When adding the third VM the communication randomly fails with destination host unreachable and

keys are still regenerated.

VM3:

1. export MKA_CAK=17a2b432c979691cdf9e3af6568171b8
2. export MKA_CKN=c640a44ca5116c584b3f278066c3ca182c9cba061681f08ee5acf9cc1d518435

1. nmcli connection add type macsec con-name macsec_test ifname macsec0 macsec.parent ens4 macsec.mode psk macsec.mka-cak $MKA_CAK macsec.mka-ckn $MKA_CKN macsec.encrypt yes ipv4.method manual ipv4.addresses 10.10.10.3/24

1. nmcli connection up macsec_test

[root@H3 ~]# ping -c1 10.10.10.1
PING 10.10.10.1 (10.10.10.1) 56(84) bytes of data.

— 10.10.10.1 ping statistics —
1 packets transmitted, 0 received, 100% packet loss, time 0ms

[root@H3 ~]# ping -c1 10.10.10.2
PING 10.10.10.2 (10.10.10.2) 56(84) bytes of data.
From 10.10.10.3 icmp_seq=1 Destination Host Unreachable

— 10.10.10.2 ping statistics —
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

Sometimes restart of wpa_supplicant and nmcli connection up/down on random VM helps to reach the

state where all three VMs works fine. But nmcli connection down/up breaks it again.

Actual results:

After ading 3-rd VM to the communication it fails randomly.

Expected results:

All three VMs are able to communicate with each other without any problems.

When I add keys manually vi pure ip cmd to all three VMs all works fine.
I mean without the support of wpa supplicant or nmcli.

Regards Michal Tesar

Additional info: