-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
4.15
-
+
-
Moderate
-
No
-
5
-
False
-
-
-
Known Issue
-
Done
Description of problem:
OCP4.15 running on top of OSP16.2 fails to configure an ipv6 ingress controller. The testcase that fails is:
make openstack-tests && KUBECONFIG=~/.kube/config ./openstack-tests run-test "[sig-installer][Suite:openshift/openstack][lb][Serial] The Openstack platform should create a TCP OVN LoadBalancer when LoadBalancerService ingressController is created on Openshift"
and it should be run with below configuration applied:
$ oc get cm -n openshift-config cloud-provider-config -o json | jq -r .data.config
[Global]
secret-name = openstack-credentials
secret-namespace = kube-system
region = regionOne
ca-file = /etc/kubernetes/static-pod-resources/configmaps/cloud-config/ca-bundle.pem
[LoadBalancer]
floating-network-id = 663be8b6-09db-4be4-bb33-7fc87c0ec51a
lb-provider = ovn
lb-method = SOURCE_IP_PORT
subnet-id = 648feab4-be61-4f69-a2a7-f54a7bbe5375
create-monitor = False
monitor-delay = 10s
monitor-timeout = 10s
monitor-max-retries = 1
max-shared-lb = 2
It's observed an error in OVN-Octavia on the controller:
: 31592, 'pool_id': '1794b0ac-44c7-4450-abad-a0725bd8a84a', 'subnet_id': '648feab4-be61-4f69-a2a7-f54a7bbe5375', 'admin_state_up': True} request_handler /usr/lib/python3.6/site-packages/networking_ovn/octavia/ovn_ driver.py:510 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver [-] Exception occurred during creation of member: netaddr.core.AddrFormatError: invalid IPNetwork [fd2e:6f44:5dd8:c956::3b4] 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver Traceback (most recent call last): 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/netaddr/strategy/ipv4.py", line 125, in str_to_int 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver return _struct.unpack('>I', _inet_pton(AF_INET, addr))[0] 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver OSError: illegal IP address string passed to inet_pton 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver During handling of the above exception, another exception occurred: 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver Traceback (most recent call last): 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/netaddr/ip/__init__.py", line 311, in __init__ 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver self._value = self._module.str_to_int(addr, flags) 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/netaddr/strategy/ipv4.py", line 129, in str_to_int 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver raise AddrFormatError('%r is not a valid IPv4 address string!' % addr) 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver netaddr.core.AddrFormatError: '[fd2e:6f44:5dd8:c956::3b4]' is not a valid IPv4 address string! 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver During handling of the above exception, another exception occurred: 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver Traceback (most recent call last): 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/netaddr/ip/__init__.py", line 791, in parse_ip_network 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver ip = IPAddress(val1, module.version, flags=INET_PTON) 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/netaddr/ip/__init__.py", line 314, in __init__ 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver % (addr, self._module.version)) 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver netaddr.core.AddrFormatError: base address '[fd2e:6f44:5dd8:c956::3b4]' is not IPv4 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver During handling of the above exception, another exception occurred: 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver Traceback (most recent call last): 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/netaddr/ip/__init__.py", line 928, in __init__ 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver implicit_prefix, flags) 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/netaddr/ip/__init__.py", line 795, in parse_ip_network 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver expanded_addr = _ipv4.expand_partial_address(val1) 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/netaddr/strategy/ipv4.py", line 259, in expand_partial_address 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver raise error 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver netaddr.core.AddrFormatError: invalid partial IPv4 address: '[fd2e:6f44:5dd8:c956::3b4]'! 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver During handling of the above exception, another exception occurred: 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver Traceback (most recent call last): 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/networking_ovn/octavia/ovn_driver.py", line 1883, in member_create 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver self._add_member(member, ovn_lb, pool_key) 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/networking_ovn/octavia/ovn_driver.py", line 1841, in _add_member 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver self._refresh_lb_vips(ovn_lb.uuid, external_ids) 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/networking_ovn/octavia/ovn_driver.py", line 1051, in _refresh_lb_vips 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver vip_ips = self._frame_vip_ips(lb_external_ids) 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/networking_ovn/octavia/ovn_driver.py", line 1039, in _frame_vip_ips 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver if netaddr.IPNetwork(lb_vip).version == 6: 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver File "/usr/lib/python3.6/site-packages/netaddr/ip/__init__.py", line 938, in __init__ 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver raise AddrFormatError('invalid IPNetwork %s' % addr) 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver netaddr.core.AddrFormatError: invalid IPNetwork [fd2e:6f44:5dd8:c956::3b4] 2024-02-06 15:28:23.072 13 ERROR networking_ovn.octavia.ovn_driver
However, this behaviour is only observed by the LB created through the openshift-ingress. Other tc that is creating a loadbalancer serviceType in ipv6 it's working fine, for example:
$ make openstack-tests && KUBECONFIG=~/.kube/config ./openstack-tests run-test "[sig-installer][Suite:openshift/openstack][lb][Serial] The Openstack platform should create a UDP OVN LoadBalancer when a UDP svc with type:LoadBalancer is created on Openshift" make: 'openstack-tests' is up to date. Feb 6 09:29:12.460: INFO: Enabling in-tree volume drivers Feb 6 09:29:12.954: INFO: configPath is now "/tmp/configfile48504311" Feb 6 09:29:12.954: INFO: The user is now "e2e-test-openstack-shwmj-user" Feb 6 09:29:12.954: INFO: Creating project "e2e-test-openstack-shwmj" Feb 6 09:29:13.275: INFO: Waiting on permissions in project "e2e-test-openstack-shwmj" ... Feb 6 09:29:13.504: INFO: Waiting for ServiceAccount "default" to be provisioned... Feb 6 09:29:13.615: INFO: Waiting for ServiceAccount "deployer" to be provisioned... Feb 6 09:29:13.736: INFO: Waiting for ServiceAccount "builder" to be provisioned... Feb 6 09:29:13.851: INFO: Waiting for RoleBinding "system:image-pullers" to be provisioned... Feb 6 09:29:13.864: INFO: Waiting for RoleBinding "system:image-builders" to be provisioned... Feb 6 09:29:13.945: INFO: Waiting for RoleBinding "system:deployers" to be provisioned... Feb 6 09:29:14.366: INFO: Project "e2e-test-openstack-shwmj" has been fully provisioned. Feb 6 09:29:15.054: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:0, Replicas:0, UpdatedReplicas:0, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:0, Conditions:[]v1.DeploymentCondition(nil), CollisionCount:(*int32)(nil)} Feb 6 09:29:17.064: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:2, UpdatedReplicas:2, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:2, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2024, time.February, 6, 9, 29, 15, 0, time.Local), LastTransitionTime:time.Date(2024, time.February, 6, 9, 29, 15, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2024, time.February, 6, 9, 29, 15, 0, time.Local), LastTransitionTime:time.Date(2024, time.February, 6, 9, 29, 15, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"lb-default-dep-749c7dd84d\" is progressing."}}, CollisionCount:(*int32)(nil)} Feb 6 09:29:19.069: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:2, UpdatedReplicas:2, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:2, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2024, time.February, 6, 9, 29, 15, 0, time.Local), LastTransitionTime:time.Date(2024, time.February, 6, 9, 29, 15, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2024, time.February, 6, 9, 29, 15, 0, time.Local), LastTransitionTime:time.Date(2024, time.February, 6, 9, 29, 15, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"lb-default-dep-749c7dd84d\" is progressing."}}, CollisionCount:(*int32)(nil)} Feb 6 09:29:21.245: INFO: Waiting up to 10m0s for service "lb-default-svc" to have a LoadBalancer Feb 6 09:29:31.554: INFO: Waiting for 6 members matching status NO_MONITOR|ONLINE in the pool pool_0_kube_service_kubernetes_e2e-test-openstack-shwmj_lb-default-svc Feb 6 09:29:47.352: INFO: Found expected number of NO_MONITOR|ONLINE members, checking again 5 seconds later to confirm that it is stable Feb 6 09:29:52.872: INFO: Wait completed, NO_MONITOR|ONLINE members found: ["ostest-qvxqg-master-2" "ostest-qvxqg-master-1" "ostest-qvxqg-worker-0-fnk9m" "ostest-qvxqg-worker-0-njvpg" "ostest-qvxqg-master-0" "ostest-qvxqg-worker-0-lmv7g"] Feb 6 09:29:53.240: INFO: Pods accessed after 100 requests: map[lb-default-dep-749c7dd84d-6j6k8:48 lb-default-dep-749c7dd84d-kjvnn:52] Feb 6 09:29:53.258: INFO: Checking if pod "lb-default-dep-749c7dd84d-6j6k8" has been accessed at least 30 number of times... Feb 6 09:29:53.258: INFO: Checking if pod "lb-default-dep-749c7dd84d-kjvnn" has been accessed at least 30 number of times... Feb 6 09:29:53.281: INFO: Deleted {user.openshift.io/v1, Resource=users e2e-test-openstack-shwmj-user}, err: <nil> Feb 6 09:29:53.331: INFO: Deleted {oauth.openshift.io/v1, Resource=oauthclients e2e-client-e2e-test-openstack-shwmj}, err: <nil> Feb 6 09:29:53.352: INFO: Deleted {oauth.openshift.io/v1, Resource=oauthaccesstokens sha256~j1Ji4c-VHrdkSG_MexljLtH4fUGLq1CDq-hwbwvJP_g}, err: <nil>
