Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-4598

[4.10z] Spike in pod-latency graph observed due to ovnkube-master restarts

XMLWordPrintable

    • Important
    • None
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      When running node-density (245 pods/node) on a 120 node cluster, we see that there is a huge spike (~22s) in Avg pod-latency. When the spike occurs we see all the ovnkube-master pods go through a restart. 

      The restart happens because of (ovnkube-master pods)

      2022-08-10T04:04:44.494945179Z panic: reflect: call of reflect.Value.Len on ptr Value

      Version-Release number of selected component (if applicable):

      4.12.0-0.nightly-2022-08-09-114621

      How reproducible:

      Steps to Reproduce:
      1. Run node-density on a 120 node cluster

      Actual results:

      Spike observed in pod-latency graph ~22s

      Expected results:

      Steady pod-latency graph ~4s

      Additional info:

              trozet@redhat.com Tim Rozet
              mohit-sheth Mohit Jitendra Sheth
              Mike Fiedler Mike Fiedler
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: