-
Bug
-
Resolution: Not a Bug
-
Undefined
-
None
-
rhel-9.6
-
None
-
No
-
Low
-
rhel-net-firewall
-
ssg_networking
-
None
-
False
-
False
-
-
None
-
None
-
None
-
None
-
Unspecified
-
Unspecified
-
Unspecified
-
-
x86_64
-
None
What were you trying to do that didn't work?
Add custom nft table, chain and rule, then start a container.
This is a simplification of a customer case in which an Openstack upgrade was failing because some of the required containers could not run.
What is the impact of this issue to you?
Worked around now by restarting the nftables service.
Please provide the package NVR for which the bug is seen:
[root@test-podman-9.6.local.lab ~]# cat /etc/redhat-release ; rpm -qf $(which nft iptables) ; podman version Red Hat Enterprise Linux release 9.6 (Plow) nftables-1.0.9-3.el9.x86_64 iptables-nft-1.8.10-11.el9_5.x86_64 Client: Podman Engine Version: 5.4.0 API Version: 5.4.0 Go Version: go1.23.4 (Red Hat 1.23.4-1.el9) Built: Wed Feb 12 10:54:13 2025 OS/Arch: linux/amd64 [root@test-podman-9.6.local.lab ~]#
How reproducible is this bug?:
Always reproducible.
Steps to reproduce
- Install iptables and nftables, run nftables service
- Add an nft table, chain and rule:
[root@test-podman-9.6.local.lab ~]# nft add table ip nat [root@test-podman-9.6.local.lab ~]# nft add chain ip nat OUTPUT [root@test-podman-9.6.local.lab ~]# nft add rule ip nat OUTPUT udp dport 6081 dnat 127.0.0.1:50000 [root@test-podman-9.6.local.lab ~]#
- Do `podman run ...`
Actual results
Podman is unable to run the container
[root@test-podman-9.6.local.lab ~]# podman run registry.redhat.io/ubi9/ubi:9.5 cat /etc/redhat-release Error: netavark: code: 1, msg: iptables: Chain already exists.[root@test-podman-9.6.local.lab ~]#
Expected results
podman can run the container, as it does when restarting nftables to remove the custom table and chain:
[root@test-podman-9.6.local.lab ~]# systemctl restart nftables [root@test-podman-9.6.local.lab ~]# podman run registry.redhat.io/ubi9/ubi:9.5 cat /etc/redhat-release Red Hat Enterprise Linux release 9.5 (Plow) [root@test-podman-9.6.local.lab ~]#
Additional information
The problem does not seem to be with podman itself. Podman is using netavark, which in turn is using the iptables command:
[root@test-podman-9.6.local.lab ~]# podman --log-level=debug run registry.redhat.io/ubi9/ubi:9.5 cat /etc/redhat-release INFO[0000] podman filtering at log level debug ... [DEBUG netavark::network::validation] Validating network namespace... [DEBUG netavark::commands::setup] Setting up... [INFO netavark::firewall] Using iptables firewall driver [DEBUG netavark::network::bridge] Setup network podman [DEBUG netavark::network::bridge] Container interface name: eth0 with IP addresses [10.88.0.39/16] [DEBUG netavark::network::bridge] Bridge name: podman0 with IP addresses [10.88.0.1/16] [DEBUG netavark::firewall::varktables::helpers] chain NETAVARK-1D8721804F16F created on table nat [DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_ISOLATION_2 exists on table filter [DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_ISOLATION_2 exists on table filter [DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_ISOLATION_3 exists on table filter [DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_ISOLATION_3 exists on table filter [DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_INPUT exists on table filter [DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_INPUT exists on table filter [DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_FORWARD exists on table filter [DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_FORWARD exists on table filter [DEBUG netavark::firewall::varktables::helpers] rule -d 10.88.0.0/16 -j ACCEPT created on table nat and chain NETAVARK-1D8721804F16F [DEBUG netavark::firewall::varktables::helpers] rule ! -d 224.0.0.0/4 -j MASQUERADE created on table nat and chain NETAVARK-1D8721804F16F [DEBUG netavark::firewall::varktables::helpers] rule -s 10.88.0.0/16 -j NETAVARK-1D8721804F16F created on table nat and chain POSTROUTING [DEBUG netavark::firewall::varktables::helpers] rule -p udp -s 10.88.0.0/16 --dport 53 -j ACCEPT created on table filter and chain NETAVARK_INPUT [DEBUG netavark::firewall::varktables::helpers] rule -p tcp -s 10.88.0.0/16 --dport 53 -j ACCEPT created on table filter and chain NETAVARK_INPUT [DEBUG netavark::firewall::varktables::helpers] rule -m conntrack --ctstate INVALID -j DROP exists on table filter and chain NETAVARK_FORWARD [DEBUG netavark::firewall::varktables::helpers] rule -d 10.88.0.0/16 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT created on table filter and chain NETAVARK_FORWARD [DEBUG netavark::firewall::varktables::helpers] rule -s 10.88.0.0/16 -j ACCEPT created on table filter and chain NETAVARK_FORWARD DEBU[0000] Unmounted container "9625b4e1b2d8a8b311bc57857e758508d0dd6529b07f64378ebd67185ca92d32" DEBU[0000] Network is already cleaned up, skipping... DEBU[0000] Cleaning up container 9625b4e1b2d8a8b311bc57857e758508d0dd6529b07f64378ebd67185ca92d32 DEBU[0000] Network is already cleaned up, skipping... DEBU[0000] Container 9625b4e1b2d8a8b311bc57857e758508d0dd6529b07f64378ebd67185ca92d32 storage is already unmounted, skipping... DEBU[0000] ExitCode msg: "netavark (exit code 1): code: 1, msg: iptables: chain already exists.\n" Error: netavark (exit code 1): code: 1, msg: iptables: Chain already exists.DEBU[0000] Shutting down engines [root@test-podman-9.6.local.lab ~]#
I'm opening this Jira for the iptables component because I see the iiptables command is showing the error above, but also in simple commands like this:
[root@test-podman-9.6.local.lab ~]# iptables -t nat -L
iptables v1.8.10 (nf_tables): table `nat' is incompatible, use 'nft' tool.
[root@test-podman-9.6.local.lab ~]#
The error is a bit misleading. If I restart nftables to remove the custom table, chain and rule that I added before, it does work, and it continues to work until I add the rule again:
[root@test-podman-9.6.local.lab ~]# systemctl restart nftables
[root@test-podman-9.6.local.lab ~]# nft add table ip nat
[root@test-podman-9.6.local.lab ~]# nft add chain ip nat OUTPUT
[root@test-podman-9.6.local.lab ~]# iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination Chain INPUT (policy ACCEPT)
target prot opt source destination Chain OUTPUT (policy ACCEPT)
target prot opt source destination Chain POSTROUTING (policy ACCEPT)
target prot opt source destination Chain OUTPUT (0 references)
target prot opt source destination
[root@test-podman-9.6.local.lab ~]#
[root@test-podman-9.6.local.lab ~]# nft add rule ip nat OUTPUT udp dport 6081 dnat 127.0.0.1:50000
[root@test-podman-9.6.local.lab ~]# iptables -t nat -L
iptables v1.8.10 (nf_tables): table `nat' is incompatible, use 'nft' tool.
[root@test-podman-9.6.local.lab ~]#
