Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-2210

ZTP with converged flow is too slow

    XMLWordPrintable

Details

    • Critical
    • 2
    • Metal Platform 226, Metal Platform 227, Metal Platform 228, Metal Platform 229, Metal Platform 230, Metal Platform 231
    • 6
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • NA

    Description

      Description of problem:

      While running scale tests with ACM provisioning 1200+ SNOs via ZTP, converged flow was enabled. With converged flow the rate at which clusters begin install is much slower than what was witnessed without converged flow.
      
      Example:
      Without converged flow - 1250/1269 SNOs completed install in 3hrs and 11m
      With converged flow - 487/1250 SNOs completed install in 10hours
      
      The test actually hit timeouts so we don't exactly know how long it took, but you can see we only managed 487 SNOs to be provisioned in 10 hours.
      
      The concurrency measurement scripts show that converged flow ran at a concurrency of 68 SNOs installing at a time vs non-converged flow peaking at 507.  Something within the converged flow is bottlenecking the SNOs install.

      Version-Release number of selected component (if applicable):

      Hub/SNO OCP 4.11.8
      ACM 2.6.1-DOWNSTREAM-2022-09-08-02-53-38

      How reproducible:

       

      Steps to Reproduce:

      1.
      2.
      3.
      

      Actual results:

       

      Expected results:

      converged flow to match previous provisioning speeds/rates

      Additional info:

      Must gather will be provided.

      Attachments

        Activity

          People

            rhn-engineering-dtantsur Dmitry Tantsur
            akrzos@redhat.com Alex Krzos
            Alex Krzos Alex Krzos
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: