Loading...

Type: Bug
Resolution: Done
Priority: Major
Fix Version/s: RHDG Operator CSV 8.5.0 GA
Affects Version/s: RHDG Operator CSV 8.5.0 GA, RHDG Operator CSV 8.4.15 GA, RHDG Operator CSV 8.4.16 GA
Component/s: Operator
Labels:
None

Blocked:
False
Blocked Reason:
None
Ready:
False
CDW devel_ack:
CDW docs_ack:
CDW pm_ack:
CDW qa_ack:
CDW release:
Target Release:

RHDG Operator CSV 8.5.0 GA
Git Pull Request:
https://github.com/infinispan/infinispan-operator/pull/2105
Intelligence Requested:
Market:

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

It's possible for Operator to get nil pointer dereference for Infinipsan cluster after Operator upgrade. In such case once user updates Infinispan cluster version the Operator will take down the cluster but doesn't bring it back up.

Even though this issue is easily workaroundable by restarting the Operator it's causing upgrade tests to be flaky.

Operator stacktrace:

2024-06-05T16:29:50.097Z	ERROR	controllers.Infinispan	goroutine 572 [running]:
runtime/debug.Stack()
	/usr/lib/golang/src/runtime/debug/stack.go:24 +0x65
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/pipeline.invokeHandler.func1()
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/pipeline/pipeline.go:55 +0xba
panic({0x179ace0, 0x29eaa90})
	/usr/lib/golang/src/runtime/panic.go:884 +0x213
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/handler/provision.probe({0x0, 0x0, 0x0, 0x0, 0x0})
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/handler/provision/pods.go:71 +0xf0
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/handler/provision.PodLivenessProbe(...)
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/handler/provision/pods.go:52
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/handler/manage.StatefulSetRollingUpgrade(0xc00080c380, {0x1c39c50, 0xc002a4c2a0})
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/handler/manage/statefulset_updates.go:95 +0x1708
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan.HandlerFunc.Handle(0x19381a0?, 0x1aa3a01?, {0x1c39c50?, 0xc002a4c2a0?})
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/api.go:40 +0x2f
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/pipeline.invokeHandler({0x1c18180?, 0x1aa3b40?}, 0xc0029a62d0?, {0x1c39c50?, 0xc002a4c2a0?})
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/pipeline/pipeline.go:60 +0x83
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/pipeline.(*impl).Process(0xc0029a6480, {0x1c2cbc8?, 0xc0029a62d0?})
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/pipeline/pipeline.go:42 +0x1a5
github.com/infinispan/infinispan-operator/controllers.(*InfinispanReconciler).Reconcile(0xc0009ce120, {0x1c2cbc8, 0xc0029a62d0}, {{{0xc002339780, 0x8}, {0xc002339776, 0xa}}})
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/controllers/infinispan.go:238 +0x354
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x1c2cb20?, {0x1c2cbc8?, 0xc0029a62d0?}, {{{0xc002339780?, 0x18e5840?}, {0xc002339776?, 0x1c1b758?}}})
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:121 +0xc8
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000458960, {0x1c2cb20, 0xc00034b5e0}, {0x17fe480?, 0xc000bd8340?})
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:320 +0x327
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000458960, {0x1c2cb20, 0xc00034b5e0})
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:273 +0x1d9
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:234 +0x85
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:230 +0x587
	{"infinispan": "datagrid/infinispan", "error": "panic occurred: runtime error: invalid memory address or nil pointer dereference"}
runtime.gopanic
	/usr/lib/golang/src/runtime/panic.go:884
runtime.panicmem
	/usr/lib/golang/src/runtime/panic.go:260
runtime.sigpanic
	/usr/lib/golang/src/runtime/signal_unix.go:841
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/handler/provision.probe
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/handler/provision/pods.go:71
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/handler/provision.PodLivenessProbe
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/handler/provision/pods.go:52
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/handler/manage.StatefulSetRollingUpgrade
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/handler/manage/statefulset_updates.go:95
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan.HandlerFunc.Handle
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/api.go:40
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/pipeline.invokeHandler
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/pipeline/pipeline.go:60
github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/pipeline.(*impl).Process
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/pkg/reconcile/pipeline/infinispan/pipeline/pipeline.go:42
github.com/infinispan/infinispan-operator/controllers.(*InfinispanReconciler).Reconcile
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/controllers/infinispan.go:238
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:121
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:320
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:273
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:234
2024-06-05T16:29:50.097Z	INFO	controllers.Infinispan	Done	{"infinispan": "datagrid/infinispan", "requeue": true, "requeueAfter": "0s", "error": "panic occurred: runtime error: invalid memory address or nil pointer dereference"}
2024-06-05T16:29:50.097Z	ERROR	Reconciler error	{"controller": "infinispan", "controllerGroup": "infinispan.org", "controllerKind": "Infinispan", "infinispan": {"name":"infinispan","namespace":"datagrid"}, "namespace": "datagrid", "name": "infinispan", "reconcileID": "043bd093-7402-457f-8e04-de155fcaf228", "error": "panic occurred: runtime error: invalid memory address or nil pointer dereference"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:273
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/remote-source/deps/gomod/src/github.com/infinispan/infinispan-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:234

links to

Upstream Issue

Details

Description

Attachments

Issue Links

Easy Agile Planning Poker

Activity

People

Dates