Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-165

Spike in pod-latency graph observed due to ovnkube-master restarts

XMLWordPrintable

    • Important
    • None
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      When running node-density (245 pods/node) on a 120 node cluster, we see that there is a huge spike (~22s) in Avg pod-latency. When the spike occurs we see all the ovnkube-master pods go through a restart. 

      The restart happens because of (ovnkube-master pods)

      2022-08-10T04:04:44.494945179Z panic: reflect: call of reflect.Value.Len on ptr Value

      Version-Release number of selected component (if applicable):

      4.12.0-0.nightly-2022-08-09-114621

      How reproducible:

      Steps to Reproduce:
      1. Run node-density on a 120 node cluster

      Actual results:

      Spike observed in pod-latency graph ~22s

      Expected results:

      Steady pod-latency graph ~4s

      Additional info:

        1. image-2022-08-16-11-26-30-614.png
          image-2022-08-16-11-26-30-614.png
          37 kB

              trozet@redhat.com Tim Rozet
              mohit-sheth Mohit Jitendra Sheth
              Mike Fiedler Mike Fiedler
              Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

                Created:
                Updated:
                Resolved: