Uploaded image for project: 'Fast Datapath Product'
  1. Fast Datapath Product
  2. FDP-1012

ovn-controller is taking lot of time to release port bindings

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Normal Normal
    • FDP-25.A
    • None
    • ovn24.09
    • 8
    • False
    • Hide

      None

      Show
      None
    • False
    • Hide

      Given a logical switch with multiple logical ports bound to a chassis,

      When all ports are deleted in a single transaction,

      Then the ovn-controller should release port bindings within a reasonable time without noticeable degradation.

      Show
      Given a logical switch with multiple logical ports bound to a chassis, When all ports are deleted in a single transaction, Then the ovn-controller should release port bindings within a reasonable time without noticeable degradation.
    • ovn24.03-24.03.4-26.el9fdp
    • rhel-9
    • rhel-sst-network-fastdatapath
    • ssg_networking
    • FDP 25.A
    • 1
    • +

      ovn-controller is taking lot of time to release port bindings if all these port bindings are 

      tracked in one engine run.


      2024-11-25T16:55:31.536Z|11341|inc_proc_eng|INFO|node: physical_flow_output, handler for input runtime_data took 920ms

      The function 
      physical_multichassis_reprocess() [1] is the culprit here. 

       

      If suppose a logical switch has 'N' logical ports and all these 'N' logical ports are bound on a chassis 'CH1' and if all the ovs ports are deleted and are handled in one engine run, then the above function iterates through all these 'N' logical ports (port bindings) N times  ( O(N * N))

      Steps to reproduce
      -------------------------

      1. Start OVN with the attached ovnnb_db.db  (a  single node is good enough)

      2. Get the names of all the logical ports of the logical switch m42-h15-000-r760

         ovn-nbctl show m42-h15-000-r760 | grep port | grep -v stor | grep -v rtos | awk '{print $2}' > ports.txt

      3.  Bind all these ports in the chassis by creating ovs ports in one transaction.

      -----------------------------------------
      #!/bin/bash
      set -x
      n="p"
      i=1
      ovs_cmd=""
      for p in $(cat ports.txt)
      do
        echo $p
        echo $i
        iface_id=$p
        name="p${i}"
        cmd=" – add-port br-int $name  -- set Interface $name external_ids:iface-id=$iface_id – set Interface $name type=internal "
        i=$((i+1))
        ovs_cmd="${ovs_cmd}${cmd}"
      done

      ovs-vsctl $ovs_cmd
      -----------------------------------------

      Run the above script.

      4. Delete all the ovs ports in one transaction and monitor the ovn-controller.log

      -----------------------------------------
      #!/bin/bash
      set -x
      n="p"
      i=1
      ovs_cmd=""
      for p in $(cat /data/ports.txt)
      do
        echo $p
        echo $i
        iface_id=$p
        name="p${i}"
        cmd=" – del-port $name "
        ovs_cmd="${ovs_cmd}${cmd}"
        i=$((i+1))
      done

      ovs-vsctl $ovs_cmd
      -----------------------------------------

      Repeat creating and deleting the ovs ports.

      You'd see deleting ports taking more time.

       

      See the corresponding OCP issue for more details - https://issues.redhat.com/browse/OCPBUGS-37825

       

      And the customer case - https://access.redhat.com/support/cases/#/case/03868814

       

       

       [1]- https://github.com/ovn-org/ovn/blob/main/controller/physical.c#L2437

       

              amusil@redhat.com Ales Musil
              nusiddiq@redhat.com Siddique Numan (Inactive)
              Jianlin Shi Jianlin Shi
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: