Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-36300

Inconsistent results and increase in p99 podLatency Ready in 4.16 Rosa HCP up to ~ 40% or above compared to 4.15 for 24 node, node-density-heavy test.

XMLWordPrintable

    • Important
    • Yes
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      Noticed increase in podLatency metrics in 4.16 Rosa HCP clusters compared to 4.15 HCP.    

      Version-Release number of selected component (if applicable):

      4.16.0-0.nightly-2024-06-27-091410    

      How reproducible:

      Rerunning tests to see if we can consistently reproduce.    

      Steps to Reproduce:

          1. Run node-density-heavy tests on 24 nodes
          2.
          3.
          

      Actual results:

      podLatency higher than expected, when comparing with 4.15    

      Consolidating all the node-density-heavy runs so far:

      4.16 worker size Ready      
          50th 99th max avg
        m5.xlarge 30000 57000 150000 20422
        m5.xlarge 15000 42000 70000 19956
        m5.xlarge 12000 42000 61000 19973
                 
        m5.2xlarge 3000 41000 42000 18375
        m5.xlarge 30000 67000 90000 21386
        m5.xlarge        
        m5.xlarge        
        m5.xlarge 30000 76000 121000 21975
                 
      4.15 m5.xlarge 8000 41000 61000 19593
        m5.xlarge 9000 41000 61000 19545
        m5.xlarge 8000 41000 51000 19500

       

      Expected results:

      4.16 podLatency should match 4.15 results    

      Additional info:

       Investigation under progress.   
      
      

            rravaiol@redhat.com Riccardo Ravaioli
            svetsa@redhat.com Sharada Vetsa
            Sharada Vetsa Sharada Vetsa
            Andrew Collins, Murali Krishnasamy
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: