Uploaded image for project: 'Fast Datapath Product'
  1. Fast Datapath Product
  2. FDP-1531 ovs-monitor-ipsec prematurely attempts to reconcile just added connections
  3. FDP-1413

[RHEL-9 OVS-3.4] ovs-monitor-ipsec prematurely attempts to reconcile just added connections

    • Icon: Sub-task Sub-task
    • Resolution: Done
    • Icon: Undefined Undefined
    • None
    • None
    • openvswitch3.4
    • None
    • 0
    • False
    • Hide

      None

      Show
      None
    • False
    • openvswitch3.4-3.4.2-70.el9fdp
    • rhel-9
    • rhel-net-ovs-dpdk
    • ssg_networking

       Problem Description: Clearly explain the issue.

      If many ipsec tunnels are addded one by one, ovs-monitor-ipsec may mark as defunct and re-create connections that it just added a second ago because they had no chance to be fully established. For example:

      2025-04-29T03:38:49Z | 871 | ovs-monitor-ipsec | INFO | Tunnel ovn-4bfa90-0 appeared in OVSDB
      2025-04-29T03:38:49Z | 873 | ovs-monitor-ipsec | INFO | Refreshing LibreSwan configuration
      2025-04-29T03:38:49Z | 877 | ovs-monitor-ipsec | INFO | Adding ipsec connection ovn-4bfa90-0-in-1
      2025-04-29T03:38:49Z | 879 | ovs-monitor-ipsec | INFO | Starting ipsec connection ovn-4bfa90-0-out-1
      2025-04-29T03:38:49Z | 881 | ovs-monitor-ipsec | INFO | Bringing up ipsec connection ovn-857210-0-in-1
      2025-04-29T03:38:49Z | 883 | ovs-monitor-ipsec | INFO | Refreshing is done.
      2025-04-29T03:38:57Z | 889 | ovs-monitor-ipsec | INFO | Tunnel ovn-cb7ce5-0 appeared in OVSDB
      2025-04-29T03:38:57Z | 891 | ovs-monitor-ipsec | INFO | Refreshing LibreSwan configuration
      2025-04-29T03:38:58Z | 895 | ovs-monitor-ipsec | INFO | Bringing up ipsec connection ovn-4bfa90-0-in-1
      2025-04-29T03:38:58Z | 897 | ovs-monitor-ipsec | INFO | Starting ipsec connection ovn-cb7ce5-0-out-1
      2025-04-29T03:38:58Z | 899 | ovs-monitor-ipsec | INFO | Adding ipsec connection ovn-cb7ce5-0-in-1
      2025-04-29T03:38:58Z | 901 | ovs-monitor-ipsec | INFO | Refreshing is done.
      

      In the log above, we bring the ovn-4bfa90-0-in-1 UP just one second after we added it, there was likely no time for the other side to initiate the connection and now we'll be initiating it from both sides creating extra possibilities for crossing streams.
       

       Impact Assessment: Describe the severity and impact (e.g., network down,availability of a workaround, etc.).

      If the crossing streams cause libreswan to enter a broken state the tunnel may be unrecoverably broken and may require manual re-creation, e.g. by restarting the affected node.
       

       Software Versions: Specify the exact versions in use (e.g.,openvswitch3.1-3.1.0-147.el8fdp).

      openvswitch3.5-3.5.0-5.el9fdp
       

        Issue Type: Indicate whether this is a new issue or a regression (if a regression, state the last known working version).

      New issue, since reconciliation logic was introduced in FDP-846.
       

       Reproducibility: Confirm if the issue can be reproduced consistently. If not, describe how often it occurs.

      100%
       

       Reproduction Steps: Provide detailed steps or scripts to replicate the issue.

      Create many ipsec tunnels one by one in a quick succession.
       

       Expected Behavior: Describe what should happen under normal circumstances.

      The 'in' connection for just added tunnel should not be brought up by ovs-monitor-ipsec for at least 15 seconds, and the connection should not be marked as defunct and re-created for another 15 seconds.
       

       Observed Behavior: Explain what actually happens.

      Connections are quickly forced to be UP and then sometimes marked as defunct and re-created just a few seconds after being added.
       

       Troubleshooting Actions: Outline the steps taken to diagnose or resolve the issue so far.

      The issue is cause by the reconciliation logic being run on every refresh, regardless of a reason for the refresh.
       

       Logs: If you collected logs please provide them (e.g. sos report, /var/log/openvswitch/* , testpmd console)

              imaximet@redhat.com Ilya Maximets
              imaximet@redhat.com Ilya Maximets
              Jiying Qiu Jiying Qiu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: