Uploaded image for project: 'OpenStack as Infra'
  1. OpenStack as Infra
  2. OSASINFRA-3973

GH-orc-587: Router newActuator silently ignores errors causing a panic

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Done
    • Icon: Undefined Undefined
    • None
    • None
    • ORC
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • Not Selected

      Originally posted on Github: https://github.com/k-orc/openstack-resource-controller/issues/587

      Problem description
      Router controller panics due to any client error in newActuator due to errors not bubbling up.
      RCA
      This issue would occur in the newActuator function any time an error returned from either clientScope, err := controller.GetScopeFactory().NewClientScopeFromObject(ctx, controller.GetK8sClient(), log, orcObject) or osClient, err := clientScope.NewNetworkClient() which might be caused by authentication error, api being unreachable etc this causes the NewNetworkClient to return an error but as the error is not reported (goes silently) in this actuator. This causes a nil osClient to be returned inside the routerActuator being returned. Which in turn would cause any router client call to return a pcnic.
      From internal/controllers/router/actuator.go:
      clientScope, err := controller.GetScopeFactory().NewClientScopeFromObject(ctx, controller.GetK8sClient(), log, orcObject) <----- any error here
      if err != nil {
      return routerActuator{}, nil <--- would cause a return of an empty osclient here
      }
      osClient, err := clientScope.NewNetworkClient() <---- any error here
      if err != nil {
      return routerActuator{}, nil <--- would cause a return of an empty osclient here
      }

      This should be fixed to be like all the other actuator and return progress.WrapError(err) instead of nil for the error value avoiding panics.
      ORC version
      687541a
      Additional information
      No response
      Relevant log output
      2025-11-26T09:55:07+02:00 ERROR Observed a panic {"controller": "router", "controllerGroup": "openstack.k-orc.cloud", "controllerKind": "Router", "Router":

      {"name":"subnet-create-full-v4","namespace":"kuttl-test-working-hedgehog"}

      , "namespace": "kuttl-test-working-hedgehog", "name": "subnet-create-full-v4", "reconcileID": "11abcf61-b568-47ad-ad92-12e09bd2bcb4", "panic": "runtime error: invalid memory address or nil pointer dereference", "panicGoValue": "\"invalid memory address or nil pointer dereference\"", "stacktrace": "goroutine 1494 [running]:\nk8s.io/apimachinery/pkg/util/runtime.logPanic(

      {0x29b8430, 0xc0013cccc0}, {0x21c0ae0, 0x3f293b0})\n\t/home/eshulman/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/runtime/runtime.go:132 +0xbc\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile.func1()\n\t/home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:198 +0x10e\npanic({0x21c0ae0?, 0x3f293b0?})\n\t/usr/lib/golang/src/runtime/panic.go:792 +0x132\ngithub.com/k-orc/openstack-resource-controller/v2/internal/controllers/router.routerActuator.ListOSResourcesForAdoption(0x0?, 0x0?, {0x29b8430?, 0xc0013cccc0?}, 0xc00048ad80?)\n\t/home/eshulman/git/openstack-resource-controller/internal/controllers/router/actuator.go:81 +0x62\ngithub.com/k-orc/openstack-resource-controller/v2/internal/controllers/generic/reconciler.getResourceForAdoption[...]({0x29b8430, 0xc0013cccc0}

      ,

      {0x7f18c46a13a8, 0xc00164f4e0}

      ,

      {0x29f1a38, 0xc000759200?}

      )\n\t/home/eshulman/git/openstack-resource-controller/internal/controllers/generic/reconciler/resource_actions.go:281 +0x8a\ngithub.com/k-orc/openstack-resource-controller/v2/internal/controllers/generic/reconciler.DeleteResource[...](

      {0x29b8430, 0xc0013cccc0}, {{0x29c14b0, 0xc0013ccdb0?}, 0xc000ae9a60?}, {0x29a4690?, 0xc00076c500}, {0x29f1a38, 0xc000759200}, {0x29b8148, ...})\n\t/home/eshulman/git/openstack-resource-controller/internal/controllers/generic/reconciler/resource_actions.go:222 +0x685\ngithub.com/k-orc/openstack-resource-controller/v2/internal/controllers/generic/reconciler.(*Controller[...]).reconcileDelete(0x29ef7e0, {0x29b8430, 0xc0013cccc0}

      ,

      {0x29f1a38, 0xc000759200}

      )\n\t/home/eshulman/git/openstack-resource-controller/internal/controllers/generic/reconciler/controller.go:265 +0x2c5\ngithub.com/k-orc/openstack-resource-controller/v2/internal/controllers/generic/reconciler.(*Controller[...]).Reconcile(0x29ef7e0,

      {0x29b8430, 0xc0013cccc0}

      , {{

      {0xc000059740, 0x0?}, {0xc00098a840?, 0x0?}}})\n\t/home/eshulman/git/openstack-resource-controller/internal/controllers/generic/reconciler/controller.go:128 +0x227\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile(0xc0013ccc30?, {0x29b8430?, 0xc0013cccc0}, {{{0xc000059740, 0x0?}

      ,

      {0xc00098a840?, 0x0?}

      }})\n\t/home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:216 +0x165\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler(0x29ecdc0,

      {0x29b8468, 0xc000476c30}, {0xc000059740, 0x1b}, {0xc00098a840, 0x15}, 0x0)\n\t/home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:461 +0x3ad\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem(0x29ecdc0, {0x29b8468, 0xc000476c30}

      )\n\t/home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:421 +0x21b\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1()\n\t/home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:296 +0x85\ncreated by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1 in goroutine 499\n\t/home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:292 +0x26b\n"}
      k8s.io/apimachinery/pkg/util/runtime.logPanic
      /home/eshulman/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/runtime/runtime.go:142
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile.func1
      /home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:198
      runtime.gopanic
      /usr/lib/golang/src/runtime/panic.go:792
      runtime.panicmem
      /usr/lib/golang/src/runtime/panic.go:262
      runtime.sigpanic
      /usr/lib/golang/src/runtime/signal_unix.go:925
      github.com/k-orc/openstack-resource-controller/v2/internal/controllers/router.routerActuator.ListOSResourcesForAdoption
      /home/eshulman/git/openstack-resource-controller/internal/controllers/router/actuator.go:81
      github.com/k-orc/openstack-resource-controller/v2/internal/controllers/generic/reconciler.getResourceForAdoption[...]
      /home/eshulman/git/openstack-resource-controller/internal/controllers/generic/reconciler/resource_actions.go:281
      github.com/k-orc/openstack-resource-controller/v2/internal/controllers/generic/reconciler.DeleteResource[...]
      /home/eshulman/git/openstack-resource-controller/internal/controllers/generic/reconciler/resource_actions.go:222
      github.com/k-orc/openstack-resource-controller/v2/internal/controllers/generic/reconciler.(*Controller[...]).reconcileDelete
      /home/eshulman/git/openstack-resource-controller/internal/controllers/generic/reconciler/controller.go:265
      github.com/k-orc/openstack-resource-controller/v2/internal/controllers/generic/reconciler.(*Controller[...]).Reconcile
      /home/eshulman/git/openstack-resource-controller/internal/controllers/generic/reconciler/controller.go:128
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile
      /home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:216
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler
      /home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:461
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem
      /home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:421
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1
      /home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:296
      2025-11-26T09:55:07+02:00 ERROR Reconciler error {"controller": "router", "controllerGroup": "openstack.k-orc.cloud", "controllerKind": "Router", "Router":

      {"name":"subnet-create-full-v4","namespace":"kuttl-test-working-hedgehog"}

      , "namespace": "kuttl-test-working-hedgehog", "name": "subnet-create-full-v4", "reconcileID": "11abcf61-b568-47ad-ad92-12e09bd2bcb4", "error": "panic: runtime error: invalid memory address or nil pointer dereference [recovered]"}
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler
      /home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:474
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem
      /home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:421
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1
      /home/eshulman/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:296

              Unassigned Unassigned
              shiftstack OpenShift On OpenStack
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: