Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-3554

ovnkube-master crash with ovn service controller panic

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • None
    • 4.8.0
    • None
    • -
    • Important
    • SDN Sprint 228, SDN Sprint 229
    • 2
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Customer Escalated

    Description

      Description of problem:

      ovnkube master container crash because service controller panic
      
      2022-11-07T04:45:39.923889951+11:00 E1106 17:45:39.923862       1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
      2022-11-07T04:45:39.923889951+11:00 goroutine 16732 [running]:
      2022-11-07T04:45:39.923889951+11:00 k8s.io/apimachinery/pkg/util/runtime.logPanic(0x190bb00, 0x28ce990)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:74 +0x95
      2022-11-07T04:45:39.923889951+11:00 k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:48 +0x86
      2022-11-07T04:45:39.923889951+11:00 panic(0x190bb00, 0x28ce990)
      2022-11-07T04:45:39.923889951+11:00     /usr/lib/golang/src/runtime/panic.go:965 +0x1b9
      2022-11-07T04:45:39.923889951+11:00 k8s.io/api/core/v1.(*Service).GetObjectKind(0x0, 0x7f25ab6ead68, 0x0)
      2022-11-07T04:45:39.923889951+11:00     <autogenerated>:1 +0x5
      2022-11-07T04:45:39.923889951+11:00 k8s.io/client-go/tools/reference.GetReference(0xc00023b420, 0x1d5f6e0, 0x0, 0x7f257803baf8, 0x0, 0x0)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/client-go/tools/reference/ref.go:59 +0x14d
      2022-11-07T04:45:39.923889951+11:00 k8s.io/client-go/tools/record.(*recorderImpl).generateEvent(0xc0078afd80, 0x1d5f6e0, 0x0, 0x0, 0xc0d21a90f70f1ab5, 0x8c5d1203c70a, 0x290c9a0, 0x1b168b6, 0x7, 0x1b36c3b, ...)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/client-go/tools/record/event.go:327 +0x5d
      2022-11-07T04:45:39.923889951+11:00 k8s.io/client-go/tools/record.(*recorderImpl).Event(0xc0078afd80, 0x1d5f6e0, 0x0, 0x1b168b6, 0x7, 0x1b36c3b, 0x1d, 0xc0167a8340, 0x186)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/client-go/tools/record/event.go:349 +0xc5
      2022-11-07T04:45:39.923889951+11:00 k8s.io/client-go/tools/record.(*recorderImpl).Eventf(0xc0078afd80, 0x1d5f6e0, 0x0, 0x1b168b6, 0x7, 0x1b36c3b, 0x1d, 0x1b7df1f, 0x41, 0xc00ef72060, ...)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/client-go/tools/record/event.go:353 +0xca
      2022-11-07T04:45:39.923889951+11:00 github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/services.(*Controller).syncServices(0xc001e60c60, 0xc00f1de210, 0x2f, 0x0, 0x0)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/services/services_controller.go:246 +0x682
      2022-11-07T04:45:39.923889951+11:00 github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/services.(*Controller).processNextWorkItem(0xc001e60c60, 0x203000)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/services/services_controller.go:184 +0xcd
      2022-11-07T04:45:39.923889951+11:00 github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/services.(*Controller).worker(...)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/services/services_controller.go:173
      2022-11-07T04:45:39.923889951+11:00 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0082cdda0)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x5f
      2022-11-07T04:45:39.923889951+11:00 k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0082cdda0, 0x1d535c0, 0xc005ef6e70, 0xc0106e2701, 0xc0000db500)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0x9b
      2022-11-07T04:45:39.923889951+11:00 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0082cdda0, 0x3b9aca00, 0x0, 0xc0106e2701, 0xc0000db500)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x98
      2022-11-07T04:45:39.923889951+11:00 k8s.io/apimachinery/pkg/util/wait.Until(0xc0082cdda0, 0x3b9aca00, 0xc0000db500)
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90 +0x4d
      2022-11-07T04:45:39.923889951+11:00 created by github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/controller/services.(*Controller).Run
      2022-11-07T04:45:39.923889951+11:00     /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/controller/services/services_controller.go:161 +0x3b1
      2022-11-07T04:45:39.927176450+11:00 panic: runtime error: invalid memory address or nil pointer dereference [recovered]
      2022-11-07T04:45:39.927176450+11:00     panic: runtime error: invalid memory address or nil pointer dereference
      2022-11-07T04:45:39.927176450+11:00 [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xd3c385]
      2022-11-07T04:45:39.927176450+11:00 
      
      
      
      
      

       

      Version-Release number of selected component (if applicable):

       

      How reproducible:

       

      Steps to Reproduce:

      1.
      2.
      3.
      

      Actual results:

       

      Expected results:

       

      Additional info:

       

       

      Attachments

        Issue Links

          Activity

            People

              pdiak@redhat.com Patryk Diak
              mmahmoud@redhat.com Mohamed Mahmoud
              Anurag Saxena Anurag Saxena
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: