-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
None
-
None
To Reproduce Steps to reproduce the behavior:
- Ensure distributed floating IP is enabled, i.e [ovn]/enable_distributed_floatin_ip=True
- Ensure tenant network types set to vlan i.e [ml2]/tenant_network_types=vlan,flat
- Create a external provider network of vlan or flat provider_network_type
- Create a router(enable-snat) with external-gateway=<external provider network> and add tenant network as subnet
- Create a tenant network and subnet
- Create 2 VMs connected to tenant network(add Floating IP to one of these in order to access) and one vm to external the provider network
- Try to ping Floating IP and VM connected to external provider network
Expected behavior
- Connectivity should work as in ml2/ovs or ml2/ovn geneve network
Actual behavior
- Connectivity broken
(overcloud) [stack@undercloud-0 ~]$ openstack network show public opensta+---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | | | created_at | 2025-05-26T11:27:29Z | | description | | | dns_domain | | | id | b30369a5-fff2-4e9b-a3e2-d4f90392c118 | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | False | | is_vlan_transparent | True | | mtu | 1500 | | name | public | | port_security_enabled | True | | project_id | 656276275e57467fb50a11faa541e5aa | | provider:network_type | vlan | | provider:physical_network | datacentre | | provider:segmentation_id | 408 | | qos_policy_id | None | | revision_number | 2 | | router:external | External | | segments | None | | shared | True | | status | ACTIVE | | subnets | 86845913-8ffe-4b94-ae7b-d6374034e5b3 | | tags | | | updated_at | 2025-05-26T11:27:37Z | +---------------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack subnet show external_subnet +----------------------+--------------------------------------+ | Field | Value | +----------------------+--------------------------------------+ | allocation_pools | 10.46.54.25-10.46.54.59 | | cidr | 10.46.54.0/26 | | created_at | 2025-05-26T11:27:37Z | | description | | | dns_nameservers | | | dns_publish_fixed_ip | None | | enable_dhcp | True | | gateway_ip | 10.46.54.62 | | host_routes | | | id | 86845913-8ffe-4b94-ae7b-d6374034e5b3 | | ip_version | 4 | | ipv6_address_mode | None | | ipv6_ra_mode | None | | name | external_subnet | | network_id | b30369a5-fff2-4e9b-a3e2-d4f90392c118 | | prefix_length | None | | project_id | 656276275e57467fb50a11faa541e5aa | | revision_number | 0 | | segment_id | None | | service_types | | | subnetpool_id | None | | tags | | | updated_at | 2025-05-26T11:27:37Z | +----------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ test -f ~/.ssh/id_rsa.pub || ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa (overcloud) [stack@undercloud-0 ~]$ openstack keypair create --public-key ~/.ssh/id_rsa.pub default # create basic security group to allow ssh/ping/dns (overcloud) [stack@undercloud-0 ~]$ openstack security group create basic # allow ssh (overcloud) [stack@undercloud-0 ~]$ openstack security group rule create basic --protocol tcp --dst-port 22:22 --remote-ip 0.0.0.0/0 # allow ping (overcloud) [stack@undercloud-0 ~]$ openstack security group rule create --protocol icmp basic # allow DNS (overcloud) [stack@undercloud-0 ~]$ openstack security group rule create --protocol udp --dst-port 53:53 basic # Create net1 of type vlan provider network (overcloud) [stack@undercloud-0 ~]$ openstack network create net1 (overcloud) [stack@undercloud-0 ~]$ openstack subnet create subnet1 --network net1 --subnet-range 192.0.2.0/24 (overcloud) [stack@undercloud-0 ~]$ openstack router create router-temp (overcloud) [stack@undercloud-0 ~]$ openstack router set router-temp --external-gateway public (overcloud) [stack@undercloud-0 ~]$ openstack router add subnet router-temp subnet1 (overcloud) [stack@undercloud-0 ~]$ openstack server create --nic net-id=net1 --image custom_neutron_guest_rhel_8.4.qcow2 --security-group basic --key-name default --flavor guest_image testvm --min 2 --max 2 # net1 is tenant vlan network (overcloud) [stack@undercloud-0 ~]$ openstack server create --nic net-id=public --image custom_neutron_guest_rhel_8.4.qcow2 --security-group basic --key-name default --flavor guest_image testvmpub (overcloud) [stack@undercloud-0 ~]$ openstack server add floating ip testvm-1 10.46.54.32 (overcloud) [stack@undercloud-0 ~]$ openstack server list --long +--------------------------------------+-----------+--------+------------+-------------+-------------------------------+-------------------------------------+--------------------------------------+-------------+-------------------+----------------------------+------------+ | ID | Name | Status | Task State | Power State | Networks | Image Name | Image ID | Flavor | Availability Zone | Host | Properties | +--------------------------------------+-----------+--------+------------+-------------+-------------------------------+-------------------------------------+--------------------------------------+-------------+-------------------+----------------------------+------------+ | a4dbbd33-15b7-46a8-8649-55955b6273f7 | testvmpub | ACTIVE | None | Running | public=10.46.54.40 | custom_neutron_guest_rhel_8.4.qcow2 | 9507a026-e85f-45ca-a6d0-7cb05f3f4897 | guest_image | nova | computesriov-1.localdomain | | | c59f4686-4d87-47c6-9b65-b81f10cf9a64 | testvm-2 | ACTIVE | None | Running | net1=192.0.2.118 | custom_neutron_guest_rhel_8.4.qcow2 | 9507a026-e85f-45ca-a6d0-7cb05f3f4897 | guest_image | nova | computesriov-0.localdomain | | | ec5a35df-6e05-4365-be57-a981980844bd | testvm-1 | ACTIVE | None | Running | net1=10.46.54.32, 192.0.2.166 | custom_neutron_guest_rhel_8.4.qcow2 | 9507a026-e85f-45ca-a6d0-7cb05f3f4897 | guest_image | nova | computesriov-1.localdomain | | +--------------------------------------+-----------+--------+------------+-------------+-------------------------------+-------------------------------------+--------------------------------------+-------------+-------------------+----------------------------+------------+ (overcloud) [stack@undercloud-0 ~]$ scp .ssh/id_rsa cloud-user@10.46.54.32:~ Warning: Permanently added '10.46.54.32' (ED25519) to the list of known hosts. id_rsa 100% 2635 4.1MB/s 00:00 (overcloud) [stack@undercloud-0 ~]$ ssh cloud-user@10.46.54.32 Warning: Permanently added '10.46.54.32' (ED25519) to the list of known hosts. Activate the web console with: systemctl enable --now cockpit.socket This system is not registered to Red Hat Insights. See https://cloud.redhat.com/ To register this system, run: insights-client --register Last login: Tue May 27 06:46:57 2025 from 10.46.4.29 [cloud-user@testvm-1 ~]$ ssh -i id_rsa 192.0.2.118 Activate the web console with: systemctl enable --now cockpit.socket This system is not registered to Red Hat Insights. See https://cloud.redhat.com/ To register this system, run: insights-client --register Last login: Tue May 27 06:49:10 2025 from 192.0.2.166 # From testvm-2 which is not connected to FIP, issue can be seen i.e it can't access FIP and VM directly connected to provider network but can access external gateway and external network like 8.8.8.8 [cloud-user@testvm-2 ~]$ ping -c 1 10.46.54.32 PING 10.46.54.32 (10.46.54.32) 56(84) bytes of data. --- 10.46.54.32 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms [cloud-user@testvm-2 ~]$ ping -c 1 10.46.54.40 PING 10.46.54.40 (10.46.54.40) 56(84) bytes of data. --- 10.46.54.40 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms [cloud-user@testvm-2 ~]$ ping -c 1 10.46.54.62 PING 10.46.54.62 (10.46.54.62) 56(84) bytes of data. 64 bytes from 10.46.54.62: icmp_seq=1 ttl=63 time=24.8 ms --- 10.46.54.62 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 24.845/24.845/24.845/0.000 ms [cloud-user@testvm-2 ~]$ ping -c 1 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=114 time=4.20 ms --- 8.8.8.8 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 4.197/4.197/4.197/0.000 ms [cloud-user@testvm-2 ~]$ logout Connection to 192.0.2.118 closed. # From testvm-1(which is connected to distributed Floating IP) it works fine, this is connected to distributed FIP [cloud-user@testvm-1 ~]$ ping -c 1 10.46.54.40 PING 10.46.54.40 (10.46.54.40) 56(84) bytes of data. 64 bytes from 10.46.54.40: icmp_seq=1 ttl=63 time=1.17 ms --- 10.46.54.40 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.174/1.174/1.174/0.000 ms [cloud-user@testvm-1 ~]$ logout Connection to 10.46.54.32 closed. (overcloud) [stack@undercloud-0 ~]$ [root@controller-0 /]# ovn-nbctl show switch 800358ad-1bb0-4011-acc4-3c0ccf4614a5 (neutron-00e3093a-0014-4be2-9de9-d874e27dca55) (aka ovn_mac_addr_net) port provnet-1c8b4529-0d92-49ff-8367-92d195af1011 type: localnet tag: 428 addresses: ["unknown"] port f3789c6c-aafd-4c3b-aa2c-96108fd2ba2b (aka computesriov-1_ovn_physnet_datacentre) addresses: ["fa:16:3e:26:16:81"] port 308fb305-b230-4d1c-bed2-017b214c7a0c type: localport addresses: ["fa:16:3e:0c:ac:71"] port ac0bcaaa-c128-454e-ade4-ba85e8ad2b45 (aka computesriov-0_ovn_physnet_datacentre) addresses: ["fa:16:3e:4a:8d:2b"] switch d336e95a-0591-4eb6-b671-bf49fd393d2b (neutron-c9995400-aad3-4508-afed-7c53362a6c22) (aka net1) port 739da310-58bd-43e2-9085-276ec50321a7 type: localport addresses: ["fa:16:3e:3f:a4:37 192.0.2.2"] port 89276377-a456-489b-a0ed-ba4242651be9 addresses: ["fa:16:3e:00:4b:b5 192.0.2.166"] port 22fda294-0432-4f5b-9f00-be20a72fb700 type: router router-port: lrp-22fda294-0432-4f5b-9f00-be20a72fb700 port 641ebfcc-5f05-4eac-b54c-5b2a3083f97c addresses: ["fa:16:3e:7d:29:b0 192.0.2.118"] port provnet-f8dbe517-64f9-4c5d-b98b-0f9ddcf020fd type: localnet tag: 434 addresses: ["unknown"] switch 4485f913-d6db-42f3-b19d-c9c0e643903b (neutron-b30369a5-fff2-4e9b-a3e2-d4f90392c118) (aka public) port dbfdef24-a996-431f-b2ce-55a9b069edde addresses: ["fa:16:3e:76:9d:a7 10.46.54.40"] port 724fa146-4d31-4b5b-ae64-354ee771d486 type: localport addresses: ["fa:16:3e:cd:08:c1 10.46.54.25"] port 16d4cccd-9fc0-40ac-9db9-0668d3f2664b type: router router-port: lrp-16d4cccd-9fc0-40ac-9db9-0668d3f2664b port provnet-2f657bee-ed30-428d-a913-76f579b9e8a6 type: localnet tag: 408 addresses: ["unknown"] router 9a1c439c-2cbc-41fa-8860-f869d826761c (neutron-348882b3-52dc-4654-8dde-c7cfbade6086) (aka router-temp) port lrp-16d4cccd-9fc0-40ac-9db9-0668d3f2664b mac: "fa:16:3e:9a:8f:5a" networks: ["10.46.54.58/26"] gateway chassis: [44e6e6d8-a620-427a-a0ca-b45f72217bad 951e938b-f77f-464f-9051-b91dcc707b59 0aee5994-648e-4c99-b301-3422048e9318] port lrp-22fda294-0432-4f5b-9f00-be20a72fb700 mac: "fa:16:3e:6e:4a:43" networks: ["192.0.2.1/24"] nat 0844e52b-b611-4d8f-b058-390a51d06f3b external ip: "10.46.54.32" logical ip: "192.0.2.166" type: "dnat_and_snat" nat ff7fc214-76db-4d51-92da-7a2f39e7aeab external ip: "10.46.54.58" logical ip: "192.0.2.0/24" type: "snat" [root@controller-0 /]# ovn-nbctl list nat _uuid : ff7fc214-76db-4d51-92da-7a2f39e7aeab allowed_ext_ips : [] exempted_ext_ips : [] external_ids : {} external_ip : "10.46.54.58" external_mac : [] external_port_range : "" gateway_port : [] logical_ip : "192.0.2.0/24" logical_port : [] options : {} type : snat _uuid : 0844e52b-b611-4d8f-b058-390a51d06f3b allowed_ext_ips : [] exempted_ext_ips : [] external_ids : {"neutron:fip_external_mac"="fa:16:3e:5a:dd:70", "neutron:fip_id"="13fc454e-d80c-484b-b652-774afce6ca67", "neutron:fip_network_id"="b30369a5-fff2-4e9b-a3e2-d4f90392c118", "neutron:fip_port_id"="89276377-a456-489b-a0ed-ba4242651be9", "neutron:revision_number"="10", "neutron:router_name"=neutron-348882b3-52dc-4654-8dde-c7cfbade6086} external_ip : "10.46.54.32" external_mac : "fa:16:3e:5a:dd:70" external_port_range : "" gateway_port : [] logical_ip : "192.0.2.166" logical_port : "89276377-a456-489b-a0ed-ba4242651be9" options : {} type : dnat_and_snat [root@controller-0 /]# ovn-sbctl --no-leader-only show Chassis "0aee5994-648e-4c99-b301-3422048e9318" hostname: controller-0.localdomain Encap geneve ip: "10.20.2.67" options: {csum="true"} Chassis "951e938b-f77f-464f-9051-b91dcc707b59" hostname: controller-2.localdomain Encap geneve ip: "10.20.2.61" options: {csum="true"} Port_Binding cr-lrp-16d4cccd-9fc0-40ac-9db9-0668d3f2664b Chassis "44e6e6d8-a620-427a-a0ca-b45f72217bad" hostname: controller-1.localdomain Encap geneve ip: "10.20.2.77" options: {csum="true"} Chassis "bf56c475-f609-4af1-9ebf-ea435a2f0def" hostname: computesriov-1.localdomain Encap geneve ip: "10.20.2.79" options: {csum="true"} Port_Binding "89276377-a456-489b-a0ed-ba4242651be9" Port_Binding "dbfdef24-a996-431f-b2ce-55a9b069edde" Chassis "5baf883b-8cf7-49c3-96c9-7c384645dc02" hostname: computesriov-0.localdomain Encap geneve ip: "10.20.2.62" options: {csum="true"} Port_Binding "641ebfcc-5f05-4eac-b54c-5b2a3083f97c"
Device Info (please complete the following information):
- Seen in Both 17.1 and 18.0, not checked 16.2 yet
Bug impact
- Users will not be able to get connectivity in above scenario
Known workaround
- None as of now
Additional context
Related slack thread https://redhat-internal.slack.com/archives/C046JULBVJ7/p1747404842118619
Attached OVN DBS and OVS DBs from gateway chassis node and both compute nodes where VMs located
