Loading...

XML

Word

Printable

Details

Type: Bug
Resolution: Done
Priority: Major
Fix Version/s: 2.12.0 GA
Affects Version/s: 2.12.0
Component/s: 3scale Operator
Labels:
- 2.12-candidate1
- cr-passed

Blocked:
False
Blocked Reason:
None
Ready:
False
3Scale PT Tested upstream:
Not Started
3scale PT Docs:
Not Started
3scale PT Product Specs:
Not Started
3scale PT Product Update Ready:
Not Started
3scale PT Released In Saas:
Not Started
3scale PT Verified Product:
Not Started
Target Release:

2.12.0 GA
Git Pull Request:
https://github.com/3scale/3scale-operator/pull/743
Steps to Reproduce:
Hide

I've created backend via CRD

connect it to one of products via CRD

delete backend usage from product and backend via UI

operator crashes:

E0427 18:26:08.111870 1 runtime.go:76] Observed a panic: Backend SystemName backend4 not found in backend index goroutine 409 [running]: k8s.io/apimachinery/pkg/util/runtime.logPanic(0x17a8ea0, 0xc000d32ba0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/runtime/runtime.go:74 +0xa3 k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/runtime/runtime.go:48 +0x82 panic(0x17a8ea0, 0xc000d32ba0) /opt/rh/go-toolset-1.13/root/usr/lib/go-toolset-1.13-golang/src/runtime/panic.go:679 +0x1b2 github.com/3scale/3scale-operator/controllers/capabilities.(*ProductThreescaleReconciler).syncBackendUsage(0xc000e6eac0, 0x0, 0x0, 0xb6868e, 0xbc8add175c44) /remote-source/app/controllers/capabilities/backend_usages.go:93 +0x164c github.com/3scale/3scale-operator/pkg/helper.(*taskRunnerImpl).Run(0xc000e6fd00, 0x1a5c53c, 0x15) /remote-source/app/pkg/helper/task_runner.go:41 +0xda github.com/3scale/3scale-operator/controllers/capabilities.(*ProductThreescaleReconciler).Reconcile(0xc000e6eac0, 0xc001ba6000, 0xc000274c90, 0xc0018ee840) /remote-source/app/controllers/capabilities/product_threescale_reconciler.go:56 +0x411 github.com/3scale/3scale-operator/controllers/capabilities.(*ProductReconciler).reconcile(0xc00000ee38, 0xc001ba6000, 0xc000b07e00, 0x1cfdc00, 0xc000866360) /remote-source/app/controllers/capabilities/product_controller.go:227 +0x3ea github.com/3scale/3scale-operator/controllers/capabilities.(*ProductReconciler).Reconcile(0xc00000ee38, 0xc0009bbad0, 0xc, 0xc0009bbab8, 0x8, 0xc000b07cc0, 0xc000842098, 0x1cbee60, 0xc000842090) /remote-source/app/controllers/capabilities/product_controller.go:149 +0x9ed sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000833dd0, 0x18930e0, 0xc0008c8660, 0xc000696d00) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:244 +0x27d sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000833dd0, 0x1b11000) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:218 +0xcb sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc000833dd0) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:197 +0x2b k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000cbd2a0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:155 +0x5e k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000cbd2a0, 0x1cabc20, 0xc000b84d50, 0x1, 0xc00065c060) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:156 +0xa3 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000cbd2a0, 0x3b9aca00, 0x0, 0x1, 0xc00065c060) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:133 +0xaa k8s.io/apimachinery/pkg/util/wait.Until(0xc000cbd2a0, 0x3b9aca00, 0xc00065c060) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:90 +0x4d created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1 /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:179 +0x44d panic: Backend SystemName backend4 not found in backend index [recovered] panic: Backend SystemName backend4 not found in backend index goroutine 409 [running]: k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/runtime/runtime.go:55 +0x105 panic(0x17a8ea0, 0xc000d32ba0) /opt/rh/go-toolset-1.13/root/usr/lib/go-toolset-1.13-golang/src/runtime/panic.go:679 +0x1b2 github.com/3scale/3scale-operator/controllers/capabilities.(*ProductThreescaleReconciler).syncBackendUsage(0xc000e6eac0, 0x0, 0x0, 0xb6868e, 0xbc8add175c44) /remote-source/app/controllers/capabilities/backend_usages.go:93 +0x164c github.com/3scale/3scale-operator/pkg/helper.(*taskRunnerImpl).Run(0xc000e6fd00, 0x1a5c53c, 0x15) /remote-source/app/pkg/helper/task_runner.go:41 +0xda github.com/3scale/3scale-operator/controllers/capabilities.(*ProductThreescaleReconciler).Reconcile(0xc000e6eac0, 0xc001ba6000, 0xc000274c90, 0xc0018ee840) /remote-source/app/controllers/capabilities/product_threescale_reconciler.go:56 +0x411 github.com/3scale/3scale-operator/controllers/capabilities.(*ProductReconciler).reconcile(0xc00000ee38, 0xc001ba6000, 0xc000b07e00, 0x1cfdc00, 0xc000866360) /remote-source/app/controllers/capabilities/product_controller.go:227 +0x3ea github.com/3scale/3scale-operator/controllers/capabilities.(*ProductReconciler).Reconcile(0xc00000ee38, 0xc0009bbad0, 0xc, 0xc0009bbab8, 0x8, 0xc000b07cc0, 0xc000842098, 0x1cbee60, 0xc000842090) /remote-source/app/controllers/capabilities/product_controller.go:149 +0x9ed sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000833dd0, 0x18930e0, 0xc0008c8660, 0xc000696d00) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:244 +0x27d sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000833dd0, 0x1b11000) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:218 +0xcb sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc000833dd0) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:197 +0x2b k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000cbd2a0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:155 +0x5e k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000cbd2a0, 0x1cabc20, 0xc000b84d50, 0x1, 0xc00065c060) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:156 +0xa3 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000cbd2a0, 0x3b9aca00, 0x0, 0x1, 0xc00065c060) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:133 +0xaa k8s.io/apimachinery/pkg/util/wait.Until(0xc000cbd2a0, 0x3b9aca00, 0xc00065c060) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:90 +0x4d created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1 /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:179 +0x44d
Show
I've created backend via CRD connect it to one of products via CRD delete backend usage from product and backend via UI operator crashes: E0427 18:26:08.111870 1 runtime.go:76] Observed a panic: Backend SystemName backend4 not found in backend index goroutine 409 [running]: k8s.io/apimachinery/pkg/util/runtime.logPanic(0x17a8ea0, 0xc000d32ba0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/runtime/runtime.go:74 +0xa3 k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/runtime/runtime.go:48 +0x82 panic(0x17a8ea0, 0xc000d32ba0) /opt/rh/go-toolset-1.13/root/usr/lib/go-toolset-1.13-golang/src/runtime/panic.go:679 +0x1b2 github.com/3scale/3scale-operator/controllers/capabilities.(*ProductThreescaleReconciler).syncBackendUsage(0xc000e6eac0, 0x0, 0x0, 0xb6868e, 0xbc8add175c44) /remote-source/app/controllers/capabilities/backend_usages.go:93 +0x164c github.com/3scale/3scale-operator/pkg/helper.(*taskRunnerImpl).Run(0xc000e6fd00, 0x1a5c53c, 0x15) /remote-source/app/pkg/helper/task_runner.go:41 +0xda github.com/3scale/3scale-operator/controllers/capabilities.(*ProductThreescaleReconciler).Reconcile(0xc000e6eac0, 0xc001ba6000, 0xc000274c90, 0xc0018ee840) /remote-source/app/controllers/capabilities/product_threescale_reconciler.go:56 +0x411 github.com/3scale/3scale-operator/controllers/capabilities.(*ProductReconciler).reconcile(0xc00000ee38, 0xc001ba6000, 0xc000b07e00, 0x1cfdc00, 0xc000866360) /remote-source/app/controllers/capabilities/product_controller.go:227 +0x3ea github.com/3scale/3scale-operator/controllers/capabilities.(*ProductReconciler).Reconcile(0xc00000ee38, 0xc0009bbad0, 0xc, 0xc0009bbab8, 0x8, 0xc000b07cc0, 0xc000842098, 0x1cbee60, 0xc000842090) /remote-source/app/controllers/capabilities/product_controller.go:149 +0x9ed sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000833dd0, 0x18930e0, 0xc0008c8660, 0xc000696d00) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:244 +0x27d sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000833dd0, 0x1b11000) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:218 +0xcb sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc000833dd0) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:197 +0x2b k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000cbd2a0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:155 +0x5e k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000cbd2a0, 0x1cabc20, 0xc000b84d50, 0x1, 0xc00065c060) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:156 +0xa3 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000cbd2a0, 0x3b9aca00, 0x0, 0x1, 0xc00065c060) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:133 +0xaa k8s.io/apimachinery/pkg/util/wait.Until(0xc000cbd2a0, 0x3b9aca00, 0xc00065c060) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:90 +0x4d created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1 /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:179 +0x44d panic: Backend SystemName backend4 not found in backend index [recovered] panic: Backend SystemName backend4 not found in backend index goroutine 409 [running]: k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/runtime/runtime.go:55 +0x105 panic(0x17a8ea0, 0xc000d32ba0) /opt/rh/go-toolset-1.13/root/usr/lib/go-toolset-1.13-golang/src/runtime/panic.go:679 +0x1b2 github.com/3scale/3scale-operator/controllers/capabilities.(*ProductThreescaleReconciler).syncBackendUsage(0xc000e6eac0, 0x0, 0x0, 0xb6868e, 0xbc8add175c44) /remote-source/app/controllers/capabilities/backend_usages.go:93 +0x164c github.com/3scale/3scale-operator/pkg/helper.(*taskRunnerImpl).Run(0xc000e6fd00, 0x1a5c53c, 0x15) /remote-source/app/pkg/helper/task_runner.go:41 +0xda github.com/3scale/3scale-operator/controllers/capabilities.(*ProductThreescaleReconciler).Reconcile(0xc000e6eac0, 0xc001ba6000, 0xc000274c90, 0xc0018ee840) /remote-source/app/controllers/capabilities/product_threescale_reconciler.go:56 +0x411 github.com/3scale/3scale-operator/controllers/capabilities.(*ProductReconciler).reconcile(0xc00000ee38, 0xc001ba6000, 0xc000b07e00, 0x1cfdc00, 0xc000866360) /remote-source/app/controllers/capabilities/product_controller.go:227 +0x3ea github.com/3scale/3scale-operator/controllers/capabilities.(*ProductReconciler).Reconcile(0xc00000ee38, 0xc0009bbad0, 0xc, 0xc0009bbab8, 0x8, 0xc000b07cc0, 0xc000842098, 0x1cbee60, 0xc000842090) /remote-source/app/controllers/capabilities/product_controller.go:149 +0x9ed sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000833dd0, 0x18930e0, 0xc0008c8660, 0xc000696d00) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:244 +0x27d sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000833dd0, 0x1b11000) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:218 +0xcb sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc000833dd0) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:197 +0x2b k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000cbd2a0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:155 +0x5e k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000cbd2a0, 0x1cabc20, 0xc000b84d50, 0x1, 0xc00065c060) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:156 +0xa3 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000cbd2a0, 0x3b9aca00, 0x0, 0x1, 0xc00065c060) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:133 +0xaa k8s.io/apimachinery/pkg/util/wait.Until(0xc000cbd2a0, 0x3b9aca00, 0xc00065c060) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.18.6/pkg/util/wait/wait.go:90 +0x4d created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1 /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.6.3/pkg/internal/controller/controller.go:179 +0x44d

SFDC Cases Counter:
SFDC Cases Links:

Description

I know that it is recommended to not modify or delete objects created by CRs from UI, API or Toolbox but user can still do this. So I think that Operator should be able to deal with this situation and do not crash because it manages other more important objects like APIManager object.

Attachments

Issue Links

links to

3scale/3scale-operator#743: Fix product backend usages controller

mentioned on

Merge request - Updated US source to: 5e0d190 Merge pull request #743 from 3scale/fix-product-backend-usages-controller

Activity

People

Assignee:: Unassigned

Reporter:: Martin Kudlej

Tester:: Martin Kudlej

Developer:: Eguzki Astiz Lezaun

Votes:: 0 Vote for this issue

Watchers:: 9 Start watching this issue

Dates

Created:: 2022/04/27 7:24 PM

Updated:: 2022/09/19 11:52 AM

Resolved:: 2022/07/25 9:37 AM