Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-2431

No change in Master node(s) memory utilization and CPU usage after replacing loaded worker nodes with larger instances on GCP

    XMLWordPrintable

Details

    • OCPNODE Sprint 226 (Blue)
    • 1
    • False
    • Hide

      None

      Show
      None

    Description

      Description of problem:

      When replacing loaded worker nodes in GCP with larger instance types (double the capacity but half the node count), running same workload scenario after worker nodes are replaced consumes more memory. Also, there are no changes/reduction in CPU consumption.

      Version-Release number of selected component (if applicable):

       

      How reproducible:

      Executing workloads before and after replacing nodes and comparing resource usage.

      Steps to Reproduce:

      1. Create a cluster with 3 master nodes ('n1-standard-4') and 30 worker nodes ('n1-standard-4') with OVN.
      2. Run kube-burner cluster-density workload (https://github.com/cloud-bulldozer/e2e-benchmarking)
      3. Note CPU and Memory resource usage of Master nodes
      4. Create a new machineset of 15 worker nodes with ('n1-standard-8') instance type.
      5. Scale down existing machineset (using 'n1-standard-4') one at a time to 0.
      6. All cluster-density pods will be successfully migrated to new machineset.
      7. Delete all cluster-density namespaces
      8. Rerun kube burner cluster-density test on the cluster with only new machineset.
      9. Collect CPU and Memory resource usage stats.
      
      

      Actual results:

      Compare results and notice increase in Memory Utlization on master nodes. Also, notice no decrease in CPU consumption.

      Expected results:

      With increased capacity of worker nodes, Memory utilization and CPU usage of master nodes is expected to decrease.

      Additional info:

      See attached spreadsheets comparing the results.
      4.9 - before vs after replacing nodes
      4.12 - before vs after replacing nodes.
      
      Cloud provider: GCP
      Results on other providers: Unavailable atm, due to infrastructure and other issues.

      Attachments

        Activity

          People

            harpatil@redhat.com Harshal Patil
            svetsa@redhat.com Sharada Vetsa
            Mallapadi Niranjan Mallapadi Niranjan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: