What were you trying to do that didn't work?

Scenario:

1. We have an ipsec.conf with a 1000 connections specified.
2. Connections have rightprotoport=udp/6081 or leftprotoport=udp/6081
3. We add each of these connections with ipsec add <conn> one by one, since there are other connections we don't want to disrupt in the process.

This is how connections are configured in OCP. 1000 connections is what we have in a 500 node cluster (500 geneve tunnels x 2 connections per tunnel (1 "in" and 1 "out")).

The process of bringing up all the 1000 connections takes about 30 minutes, because one ipsec add takes roughly 2 seconds.

More details, analysis and the test script, as well as a potential solution for this problem, are available in this upstream PR:
https://github.com/libreswan/libreswan/pull/1987

This issue is open to facilitate backporting of these changes once they are accepted upstream, or implement alternative solution if they will not be accepted.

What is the impact of this issue to you?

If pluto crashes and ipsec pod is restarted, it takes 30 minutes to restore connectivity between nodes. Same is after reboot or after initial enabling of ipsec in the cluster.

Please provide the package NVR for which the bug is seen:

Any current version, but, to be specific, let's say libreswan-4.6-3.el9_0.3.

How reproducible is this bug?: 100%

Steps to reproduce

1.  Create an ipsec.conf file with 1000 connections, each of which has rightprotoport=udp/6081 or leftprotoport=udp/6081.
2.  Try to add them all one by one with ipsec add.

Expected results

Expected a reasonable time for completion. Under 1 minute may be fine.

Actual results

Takes 30 minutes.