Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-6798

ovn-operator crashes during greenfield deployment

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Normal Normal
    • rhos-18.0.0
    • None
    • ovn-operator
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • ?
    • ovn-operator-container-1.0.0-46
    • None
    • Low

      2024-05-07T08:15:44.524Z	INFO	Controllers.OVNController	OVS DaemonSet not ready yet. Configuration job cannot be started.	{"controller": "ovncontroller", "controllerGroup": "ovn.openstack.org", "controllerKind": "OVNController", "OVNController": {"name":"ovncontroller","namespace":"openstack"}, "namespace": "openstack", "name": "ovncontroller", "reconcileID": "d15d4c3c-290b-4cc0-8eae-2345798fcd47"}
      2024-05-07T08:15:44.539Z	INFO	Observed a panic in reconciler: runtime error: slice bounds out of range [:1] with capacity 0	{"controller": "ovndbcluster", "controllerGroup": "ovn.openstack.org", "controllerKind": "OVNDBCluster", "OVNDBCluster": {"name":"ovndbcluster-sb","namespace":"openstack"}, "namespace": "openstack", "name": "ovndbcluster-sb", "reconcileID": "3089ad8f-bbb4-4d02-8741-aa3d63b4c2bc"}
      panic: runtime error: slice bounds out of range [:1] with capacity 0 [recovered]
      	panic: runtime error: slice bounds out of range [:1] with capacity 0
      
      goroutine 419 [running]:
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
      	/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.5/pkg/internal/controller/controller.go:116 +0x1fa
      panic({0x1b128e0, 0xc004b7f2f0})
      	/usr/lib/golang/src/runtime/panic.go:884 +0x213
      github.com/openstack-k8s-operators/ovn-operator/controllers.(*OVNDBClusterReconciler).reconcileServices(0x1ebe3c0?, {0x1ebe3c0, 0xc0045a3140}, 0xc00457c900, 0xc003cc2790, 0xc004751650, {0x1c30952, 0xe})
      	/remote-source/controllers/ovndbcluster_controller.go:723 +0x10c5
      github.com/openstack-k8s-operators/ovn-operator/controllers.(*OVNDBClusterReconciler).reconcileNormal(0x1ed4788?, {0x1ebe3c0, 0xc0045a3140}, 0xc00457c900, 0xc003cc2790)
      	/remote-source/controllers/ovndbcluster_controller.go:544 +0xea9
      github.com/openstack-k8s-operators/ovn-operator/controllers.(*OVNDBClusterReconciler).Reconcile(0xc00052d200, {0x1ebe3c0, 0xc0045a3140}, {{{0xc00459ebd0?, 0x5?}, {0xc00459ebc0?, 0xc0051afd48?}}})
      	/remote-source/controllers/ovndbcluster_controller.go:204 +0xfc9
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x1ec0a00?, {0x1ebe3c0?, 0xc0045a3140?}, {{{0xc00459ebd0?, 0xb?}, {0xc00459ebc0?, 0x0?}}})
      	/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.5/pkg/internal/controller/controller.go:119 +0xc8
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000515360, {0x1ebe318, 0xc0003458b0}, {0x1a7d5a0?, 0xc00038ece0?})
      	/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.5/pkg/internal/controller/controller.go:316 +0x3ca
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000515360, {0x1ebe318, 0xc0003458b0})
      	/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.5/pkg/internal/controller/controller.go:266 +0x1d9
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
      	/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.5/pkg/internal/controller/controller.go:227 +0x85
      created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
      	/opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.5/pkg/internal/controller/controller.go:223 +0x587
      
      

      After the crash the new operator instance seems to be stable and the deployment finishes. So this is probably a race condition that producing an unhandled edge case.

              averdagu@redhat.com Arnau Verdaguer Puigdollers
              rh-ee-bgibizer Balazs Gibizer
              Maor Blaustein Maor Blaustein
              rhos-dfg-networking-squad-neutron
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: