Uploaded image for project: 'Maistra'
  1. Maistra
  2. MAISTRA-2371

Handle tombstones in listerInformer

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Critical
    • Resolution: Done
    • Affects Version/s: maistra-1.1.13
    • Fix Version/s: maistra-1.1.15
    • Component/s: None
    • Labels:
      None

      Description

      The updated cache codebase is not handling tombstones when translating the events from the namespace caches to the aggregated cache, leading to a panic in the goroutine similar to the following.

      2021-05-18T22:20:16.658766Z	error	Observed a panic: &runtime.TypeAssertionError{_interface:(*runtime._type)(nil), concrete:(*runtime._type)(0x1e5b8c0), asserted:(*runtime._type)(0x1e33be0), missingMethod:"DeepCopyObject"} (interface conversion: cache.DeletedFinalStateUnknown is not runtime.Object: missing method DeepCopyObject)
      goroutine 18 [running]:
      k8s.io/apimachinery/pkg/util/runtime.logPanic(0x1e28f40, 0xc00035eab0)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/runtime/runtime.go:74 +0xa6
      k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/runtime/runtime.go:48 +0x89
      panic(0x1e28f40, 0xc00035eab0)
      	/usr/local/go/src/runtime/panic.go:969 +0x1b9
      istio.io/istio/pkg/listwatch.(*listerInformer).newWatchEvent(...)
      	/Users/kevin/istio/build/source/istio/pkg/listwatch/listinformer.go:138
      istio.io/istio/pkg/listwatch.(*listerInformer).OnDelete(0xc000349020, 0x1e5b8c0, 0xc00041c060)
      	/Users/kevin/istio/build/source/istio/pkg/listwatch/listinformer.go:163 +0x45
      k8s.io/client-go/tools/cache.(*processorListener).run.func1()
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/shared_informer.go:746 +0x166
      k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0000a8760)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/wait/wait.go:155 +0x5f
      k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000118f60, 0x20bfc20, 0xc000112000, 0x1e03b01, 0xc000110000)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/wait/wait.go:156 +0xad
      k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0000a8760, 0x3b9aca00, 0x0, 0x1, 0xc000110000)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/wait/wait.go:133 +0x98
      k8s.io/apimachinery/pkg/util/wait.Until(...)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/wait/wait.go:90
      k8s.io/client-go/tools/cache.(*processorListener).run(0xc000342980)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/shared_informer.go:738 +0x95
      k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc00036c060, 0xc000380000)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/wait/wait.go:73 +0x51
      created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/wait/wait.go:71 +0x65
      
      panic: interface conversion: cache.DeletedFinalStateUnknown is not runtime.Object: missing method DeepCopyObject [recovered]
      	panic: interface conversion: cache.DeletedFinalStateUnknown is not runtime.Object: missing method DeepCopyObject
      
      goroutine 18 [running]:
      k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/runtime/runtime.go:55 +0x10c
      panic(0x1e28f40, 0xc00035eab0)
      	/usr/local/go/src/runtime/panic.go:969 +0x1b9
      istio.io/istio/pkg/listwatch.(*listerInformer).newWatchEvent(...)
      	/Users/kevin/istio/build/source/istio/pkg/listwatch/listinformer.go:138
      istio.io/istio/pkg/listwatch.(*listerInformer).OnDelete(0xc000349020, 0x1e5b8c0, 0xc00041c060)
      	/Users/kevin/istio/build/source/istio/pkg/listwatch/listinformer.go:163 +0x45
      k8s.io/client-go/tools/cache.(*processorListener).run.func1()
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/shared_informer.go:746 +0x166
      k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0000a8760)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/wait/wait.go:155 +0x5f
      k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0001c7f60, 0x20bfc20, 0xc000112000, 0x1e03b01, 0xc000110000)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/wait/wait.go:156 +0xad
      k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0000a8760, 0x3b9aca00, 0x0, 0x1, 0xc000110000)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/wait/wait.go:133 +0x98
      k8s.io/apimachinery/pkg/util/wait.Until(...)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/wait/wait.go:90
      k8s.io/client-go/tools/cache.(*processorListener).run(0xc000342980)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/shared_informer.go:738 +0x95
      k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc00036c060, 0xc000380000)
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/wait/wait.go:73 +0x51
      created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start
      	/Users/kevin/istio/istio-all/gopath/pkg/mod/k8s.io/apimachinery@v0.18.4/pkg/util/wait/wait.go:71 +0x65
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              kconner@redhat.com Kevin Conner
              Reporter:
              kconner@redhat.com Kevin Conner
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: