Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-14285

[OCP 4.14] High p99 podready latency in OVN-IC with Incremental Processing

XMLWordPrintable

    • No
    • SDN Sprint 237
    • 1
    • Approved
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      We see 19 minutes p99 podready latency for node-density light workload on OVN-IC 120 node environment with Northd Incremental Processing patches i.e
OVN_IMAGE - quay.io/itssurya/dev-images:ic-scale-ovnk-v2
CNO image - quay.io/itssurya/dev-images:ic-cno-hack
I have got high latency with northd single thread CNO image ( quay.io/itssurya/dev-images:ic-cno-hack-1thread ) as well

Slides for this testing https://docs.google.com/presentation/d/16NMDoF52gUb-MybbAtSs16_888nLbshQYCU1Zw3WuvE/edit#slide=id.g24a34730293_0_0

Version-Release number of selected component (if applicable):
OCP 4.14
OVN_IMAGE - quay.io/itssurya/dev-images:ic-scale-ovnk-v2
CNO image - quay.io/itssurya/dev-images:ic-cno-hack

      Version-Release number of selected component (if applicable):

      OCP 4.14
OVN_IMAGE - quay.io/itssurya/dev-images:ic-scale-ovnk-v2
CNO image - quay.io/itssurya/dev-images:ic-cno-hack

      How reproducible:

       

      Steps to Reproduce:

      1. Deploy 120 node OCP environment on AWS
2. Apply OVN builds
3. Run node-denisty (light) and cluster-density-v2 workloads

      Actual results:

      p99 pod ready latency is 19 minutes

      Expected results:

      p99 pod ready latency is 1.77 seconds in OVN_IC without Incremental processing

      Additional info:

       

              npinaeva@redhat.com Nadia Pinaeva
              vkommadi@redhat.com VENKATA ANIL kumar KOMMADDI
              Anurag Saxena Anurag Saxena
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: