Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-55741

During RA creation at scale, 800% CPU usage by ovnkube-controller

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None
    • Rejected
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      When the scale tests create 72 RouteAdvertisements with qps=20 and burst=20 with 1:1 RA:CUDN, ovnkue-controller CPU uage is going above 800%. This is observed in all nodes (its a 18 worker node env)

       I have attached the pprof, audit logs and must-gather which captured during the testing.

      Audit logs https://storage.scalelab.redhat.com/anilvenkata/bgp/rahighcpu/must-gather.local.7838236669567166586.tar.gz 

      must-gather https://storage.scalelab.redhat.com/anilvenkata/bgp/rahighcpu/must-gather.local.581736730274221847.tar.gz 

      RA configuration https://storage.scalelab.redhat.com/anilvenkata/bgp/rahighcpu/log_routeadvertisement_all.yaml 
      CPU pprof https://storage.scalelab.redhat.com/anilvenkata/bgp/rahighcpu/pprof/ovnkube-cpu1.pprof 

      Memoy pprof https://storage.scalelab.redhat.com/anilvenkata/bgp/rahighcpu/pprof/ovnkube-heap1.pprof 

      This is a follow up of the bug https://issues.redhat.com/browse/OCPBUGS-54375 

      Dashboard showing 800% cpu usage by ovnkube-controller

              sdn-team-bot sdn-team bot
              vkommadi@redhat.com VENKATA ANIL kumar KOMMADDI
              None
              None
              Anurag Saxena Anurag Saxena
              None
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: