Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-1012

Spike in pod-latency graph observed due to ovnkube-master restarts

XMLWordPrintable

    • Important
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      When running node-density (245 pods/node) on a 120 node cluster, we see that there is a huge spike (~22s) in Avg pod-latency. When the spike occurs we see all the ovnkube-master pods go through a restart. 

      The restart happens because of (ovnkube-master pods)

      2022-08-10T04:04:44.494945179Z panic: reflect: call of reflect.Value.Len on ptr Value

      Version-Release number of selected component (if applicable):

      4.12.0-0.nightly-2022-08-09-114621

      How reproducible:

      Steps to Reproduce:
      1. Run node-density on a 120 node cluster

      Actual results:

      Spike observed in pod-latency graph ~22s

      Expected results:

      Steady pod-latency graph ~4s

      Additional info:

            trozet@redhat.com Tim Rozet
            mohit-sheth Mohit Jitendra Sheth
            Michael Fiedler Michael Fiedler
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: