Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-17782

[OVN-IC] Regression in Ovnkube-node container memory usage (380% increase)

XMLWordPrintable

    • No
    • SDN Sprint 242
    • 1
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Ovnkube-node container max memory usage was 110 MiB with 4.14.0-0.nightly-2023-05-18-231932 image and now it is 530 MiB with 4.14.0-0.nightly-2023-07-31-181848 image, for the same test (cluster-density-v2 with 800 iterations, churn=false) on 120 node environment. We observed the same pattern in the OVN-IC environment as well.

      Note: As churn is false, we are calculating memory usage for only resource creation.

      Grafana panel for OVN with 4.14.0-0.nightly-2023-05-18-231932 image -

      https://grafana.rdu2.scalelab.redhat.com:3000/dashboard/snapshot/H9pAb07fsPEOFyd5dhKLFP602A7S18uC

       

      Grafana panel for OVN with 4.14.0-0.nightly-2023-07-31-181848 image -

      https://grafana.rdu2.scalelab.redhat.com:3000/dashboard/snapshot/8158bJgv3e4P2uiVernbc2E5ypBWFYHt

       

      As the test was successfully run in the CI, we couldn't collect a must-gather. I can provide must-gather and pprof data if needed.

       

      We observed 100 MiB to 550 MiB increase in OVN-IC between 4.14.0-0.nightly-2023-06-12-141936 and  4.14.0-0.nightly-2023-07-30-191504 versions.

      OVN-IC  4.14.0-0.nightly-2023-06-12-141936

      https://grafana.rdu2.scalelab.redhat.com:3000/dashboard/snapshot/o5SXLdHIL8whsdgaMyXwWamipBP8J2fF

       

      OVN-IC 4.14.0-0.nightly-2023-07-30-191504

      https://grafana.rdu2.scalelab.redhat.com:3000/dashboard/snapshot/NMuSQx7YAJ9jokoKMl6Me9StHp33tjwD

            npinaeva@redhat.com Nadia Pinaeva
            vkommadi@redhat.com VENKATA ANIL kumar KOMMADDI
            Anurag Saxena Anurag Saxena
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: