Description of problem:
NNCP rollout completes, (verified via manual check of interfaces on nodes) but reports back as ConfigurationAborted when reviewing NNCE objects.
Version-Release number of selected component (if applicable):
How reproducible: Easy
Steps to Reproduce:{code:none}
1. Apply NNCE
2. Wait for completion, find operators in {STATUS: Aborted}
3. Delete NNCE, issue moves from BONDX to BONDY on same host
Actual results:
worker-64.kub11.rchltxek.vzwops.com.wsnmacvlanpolicy-bond1 Available SuccessfullyConfigured
worker-64.kub11.rchltxek.vzwops.com.wsnmacvlanpolicy-bond2 Aborted ConfigurationAborted
Expected results:{code:none} worker-64.kub11.rchltxek.vzwops.com.wsnmacvlanpolicy-bond1 Available SuccessfullyConfigured worker-64.kub11.rchltxek.vzwops.com.wsnmacvlanpolicy-bond2 Available
Additional info:
These errors seem to indicate it cannot reduce the available node count. (taken from the nmstate-operator pods) ``` 2024-10-07T21:30:47.477770344Z {"level":"error","ts":"2024-10-07T21:30:47.477Z","logger":"controllers.NodeNetworkConfigurationPolicy","msg":"error decrementing unavailableNodeCount with cached client, trying again with non-cached client.","error":"no unavailable nodes","stacktrace":"github.com/nmstate/kubernetes-nmstate/controllers/handler.(*NodeNetworkConfigurationPolicyReconciler).decrementUnavailableNodeCount\n\t/go/src/github.com/openshift/kubernetes-nmstate/controllers/handler/nodenetworkconfigurationpolicy_controller.go:447\ngithub.com/nmstate/kubernetes-nmstate/controllers/handler.(*NodeNetworkConfigurationPolicyReconciler).Reconcile\n\t/go/src/github.com/openshift/kubernetes-nmstate/controllers/handler/nodenetworkconfigurationpolicy_controller.go:231\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/src/github.com/openshift/kubernetes-nmstate/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/src/github.com/openshift/kubernetes-nmstate/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/kubernetes-nmstate/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/src/github.com/openshift/kubernetes-nmstate/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:235"} 2024-10-07T21:30:47.482625750Z {"level":"error","ts":"2024-10-07T21:30:47.482Z","logger":"controllers.NodeNetworkConfigurationPolicy","msg":"error decrementing unavailableNodeCount with non-cached client","error":"no unavailable nodes","stacktrace":"github.com/nmstate/kubernetes-nmstate/controllers/handler.(*NodeNetworkConfigurationPolicyReconciler).decrementUnavailableNodeCount\n\t/go/src/github.com/openshift/kubernetes-nmstate/controllers/handler/nodenetworkconfigurationpolicy_controller.go:450\ngithub.com/nmstate/kubernetes-nmstate/controllers/handler.(*NodeNetworkConfigurationPolicyReconciler).Reconcile\n\t/go/src/github.com/openshift/kubernetes-nmstate/controllers/handler/nodenetworkconfigurationpolicy_controller.go:231\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/src/github.com/openshift/kubernetes-nmstate/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/src/github.com/openshift/kubernetes-nmstate/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/kubernetes-nmstate/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/src/github.com/openshift/kubernetes-nmstate/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:235"} 2024-10-07T21:30:49.552115207Z {"level":"error","ts":"2024-10-07T21:30:49.552Z","logger":"controllers.NodeNetworkConfigurationPolicy","msg":"","nodenetworkconfigurationpolicy":{"name":"wsnmacvlanpolicy-bond2"},"error":"policy has failing enactments, aborting","stacktrace":"github.com/nmstate/kubernetes-nmstate/controllers/handler.(*NodeNetworkConfigurationPolicyReconciler).Reconcile\n\t/go/src/github.com/openshift/kubernetes-nmstate/controllers/handler/nodenetworkconfigurationpolicy_controller.go:195\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/src/github.com/openshift/kubernetes-nmstate/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/src/github.com/openshift/kubernetes-nmstate/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/kubernetes-nmstate/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/src/github.com/openshift/kubernetes-nmstate/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:235"} ```
SF Ticket Ref: 03941185