-
Bug
-
Resolution: Done
-
Normal
-
None
-
4.12
-
Quality / Stability / Reliability
-
False
-
-
None
-
Moderate
-
None
-
None
-
None
-
ShiftStack Sprint 227, ShiftStack Sprint 228
-
2
-
+
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
While creating a LoadBalancer service with ETP:Local and monitors are enabled, the healtmonitor is first created and then unexpectedly removed. As a result, the loadbalancer remained in PENDING_UPDATE status indefinitely.
Version-Release number of selected component (if applicable):
OCP4.12.0-0.nightly-2022-11-07-181244 on top of RHOS-17.0-RHEL-9-20220909.n.0 with OVN-Provider and monitors enabled.
How reproducible:
Always
Steps to Reproduce:
$ oc get cm/cloud-provider-config -n openshift-config -o yaml
apiVersion: v1
data:
[...]
config: |
[Global]
secret-name = openstack-credentials
secret-namespace = kube-system
region = regionOne
ca-file = /etc/kubernetes/static-pod-resources/configmaps/cloud-config/ca-bundle.pem
[LoadBalancer]
lb-provider=ovn
lb-method=SOURCE_IP_PORT
create-monitor=true
monitor-delay=60s
monitor-timeout=30s
monitor-max-retries=5
kind: ConfigMap
$ oc apply -f ETPlocal-manifests.yaml
project.project.openshift.io/udp-lb-etplocal-ns created
deployment.apps/udp-lb-etplocal-dep created
service/udp-lb-etplocal-svc created
$ openstack loadbalancer healthmonitor list
+--------------------------------------+-----------------------------------------------------------------------------+----------------------------------+-------------+----------------+
| id | name | project_id | type | admin_state_up |
+--------------------------------------+-----------------------------------------------------------------------------+----------------------------------+-------------+----------------+
| b76aa733-43e7-4aab-a02f-b99f78679228 | monitor_8082_kube_service_kubernetes_udp-lb-etplocal-ns_udp-lb-etplocal-svc | 08b7f0b053904f16a7d65496d21f3efc | UDP-CONNECT | True |
+--------------------------------------+-----------------------------------------------------------------------------+----------------------------------+-------------+----------------+
$ openstack loadbalancer healthmonitor list
+--------------------------------------+-----------------------------------------------------------------------------+----------------------------------+-------------+----------------+
| id | name | project_id | type | admin_state_up |
+--------------------------------------+-----------------------------------------------------------------------------+----------------------------------+-------------+----------------+
| b76aa733-43e7-4aab-a02f-b99f78679228 | monitor_8082_kube_service_kubernetes_udp-lb-etplocal-ns_udp-lb-etplocal-svc | 08b7f0b053904f16a7d65496d21f3efc | UDP-CONNECT | True |
+--------------------------------------+-----------------------------------------------------------------------------+----------------------------------+-------------+----------------+
$ openstack loadbalancer healthmonitor list
$ openstack loadbalancer list
+--------------------------------------+----------------------------------------------------------------+----------------------------------+--------------+---------------------+------------------+----------+
| id | name | project_id | vip_address | provisioning_status | operating_status | provider |
+--------------------------------------+----------------------------------------------------------------+----------------------------------+--------------+---------------------+------------------+----------+
| d2b2b376-cfc5-4fd2-a2a8-738c28e0b878 | kube_service_kubernetes_udp-lb-etplocal-ns_udp-lb-etplocal-svc | 08b7f0b053904f16a7d65496d21f3efc | 10.196.1.246 | PENDING_UPDATE | ONLINE | ovn |
+--------------------------------------+----------------------------------------------------------------+----------------------------------+--------------+---------------------+------------------+----------+
$ oc logs -n openshift-cloud-controller-manager openstack-cloud-controller-manager-db4b75f6b-7dsb8 -f
[...]
I1111 11:24:50.490982 1 event.go:294] "Event occurred" object="udp-lb-etplocal-ns/udp-lb-etplocal-svc" fieldPath="" kind="Service" apiVersion="v1" type="Normal" reason="EnsuringLoadBalancer" message="Ensuring load balancer"
I1111 11:24:50.561299 1 loadbalancer.go:2153] "EnsureLoadBalancer" cluster="kubernetes" service="udp-lb-etplocal-ns/udp-lb-etplocal-svc"
W1111 11:24:50.844229 1 loadbalancer.go:1893] LoadBalancerSourceRanges is ignored
I1111 11:24:50.884179 1 loadbalancer.go:2044] "Creating fully populated loadbalancer" lbName="kube_service_kubernetes_udp-lb-etplocal-ns_udp-lb-etplocal-svc" service="udp-lb-etplocal-ns/udp-lb-etplocal-svc"
I1111 11:24:56.835103 1 loadbalancer.go:163] "Waiting for load balancer ACTIVE" lbID="d2b2b376-cfc5-4fd2-a2a8-738c28e0b878"
I1111 11:24:56.879839 1 loadbalancer.go:177] "Load balancer ACTIVE" lbID="d2b2b376-cfc5-4fd2-a2a8-738c28e0b878"
I1111 11:24:58.127228 1 event.go:294] "Event occurred" object="udp-lb-etplocal-ns/udp-lb-etplocal-svc" fieldPath="" kind="Service" apiVersion="v1" type="Normal" reason="EnsuredLoadBalancer" message="Ensured load balancer"
I1111 11:24:58.151021 1 loadbalancer.go:2153] "EnsureLoadBalancer" cluster="kubernetes" service="udp-lb-etplocal-ns/udp-lb-etplocal-svc"
I1111 11:24:58.151391 1 event.go:294] "Event occurred" object="udp-lb-etplocal-ns/udp-lb-etplocal-svc" fieldPath="" kind="Service" apiVersion="v1" type="Normal" reason="EnsuringLoadBalancer" message="Ensuring load balancer"
W1111 11:24:58.469438 1 loadbalancer.go:1893] LoadBalancerSourceRanges is ignored
I1111 11:25:02.738876 1 loadbalancer.go:163] "Waiting for load balancer ACTIVE" lbID="d2b2b376-cfc5-4fd2-a2a8-738c28e0b878"
I1111 11:25:02.774938 1 loadbalancer.go:177] "Load balancer ACTIVE" lbID="d2b2b376-cfc5-4fd2-a2a8-738c28e0b878"
I1111 11:25:02.815139 1 loadbalancer.go:1181] "Recreating health monitor for the pool" pool="7f2981d0-39cd-4adf-8d2d-5e3b5f808099" oldMonitor="b76aa733-43e7-4aab-a02f-b99f78679228"
I1111 11:25:03.734448 1 loadbalancer.go:163] "Waiting for load balancer ACTIVE" lbID="d2b2b376-cfc5-4fd2-a2a8-738c28e0b878"
Actual results:
Loadbalancer in PENDING_UPDATE status, cannot be removed, so the OCP resource cannot be deleted either.
Expected results:
Succesful loadbalancer svc creation and operative