------- Comment From Chakravarthi@ibm.com 2025-03-18 10:16 EDT-------
Verified on the latest 9.6 nightly build (5.14.0-540) which has leaap repository version (0.22.0-1)

[root@za93ka-sanity-provision-small-qcow-10-20-113-119 network-scripts]# hostnamectl
Static hostname: za93ka-sanity-provision-small-qcow-10-20-113-119
Icon name: computer-vm
Chassis: vm ??
Machine ID: 872a8ee8045047b3a2c13118f58efd6b
Boot ID: 26f801b9f80d446ab100657774173b84
Virtualization: kvm
Operating System: Red Hat Enterprise Linux 9.6 Beta (Plow)
CPE OS Name: cpe:/o:redhat:enterprise_linux:9::baseos
Kernel: Linux 5.14.0-570.el9.s390x
Architecture: s390x

It raises a inhibitor when legacy network config is present..

============================================================
REPORT OVERVIEW
============================================================

Upgrade has been inhibited due to the following problems:
1. Legacy network configuration found

HIGH and MEDIUM severity reports:
1. Berkeley DB (libdb) has been detected on your system

Reports summary:
Errors: 0
Inhibitors: 1
HIGH severity reports: 0
MEDIUM severity reports: 1
LOW severity reports: 2
INFO severity reports: 2

Before continuing, review the full report below for details about discovered problems and possible remediation instructions:
A report has been generated at /var/log/leapp/leapp-report.txt
A report has been generated at /var/log/leapp/leapp-report.json

============================================================
END OF REPORT OVERVIEW
============================================================

Upon migrating the legacy network using "nmcli con migrate", inhibitor was resolved and upgrade also went through..

============================================================
REPORT OVERVIEW
============================================================

HIGH and MEDIUM severity reports:
1. Berkeley DB (libdb) has been detected on your system

Reports summary:
Errors: 0
Inhibitors: 0
HIGH severity reports: 0
MEDIUM severity reports: 1
LOW severity reports: 3
INFO severity reports: 2

Before continuing, review the full report below for details about discovered problems and possible remediation instructions:
A report has been generated at /var/log/leapp/leapp-report.txt
A report has been generated at /var/log/leapp/leapp-report.json

============================================================
END OF REPORT OVERVIEW
============================================================

[rundeck@za93ka-sanity-provision-small-qcow-10-20-113-119 ~]$ hostnamectl
Static hostname: za93ka-sanity-provision-small-qcow-10-20-113-119
Icon name: computer-vm
Chassis: vm ??
Machine ID: 872a8ee8045047b3a2c13118f58efd6b
Boot ID: 130e684c61dc45b4807a3ceafce3743a
Virtualization: kvm
Operating System: Red Hat Enterprise Linux 10.0 Beta (Coughlan)
CPE OS Name: cpe:/o:redhat:enterprise_linux:10::baseos
Kernel: Linux 6.12.0-55.1.1.el10_0.s390x
Architecture: s390x

Also, network came up with eth0 post upgrade:

[rundeck@za93ka-sanity-provision-small-qcow-10-20-113-119 ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:79:41:a8 brd ff:ff:ff:ff:ff:ff
altname enc1
altname enx5254007941a8
inet 10.20.113.119/22 brd 10.20.115.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe79:41a8/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever

[rundeck@za93ka-sanity-provision-small-qcow-10-20-113-119 ~]$ hostnamectl
Static hostname: za93ka-sanity-provision-small-qcow-10-20-113-119
Icon name: computer-vm
Chassis: vm ??
Machine ID: 872a8ee8045047b3a2c13118f58efd6b
Boot ID: 130e684c61dc45b4807a3ceafce3743a
Virtualization: kvm
Operating System: Red Hat Enterprise Linux 10.0 Beta (Coughlan)
CPE OS Name: cpe:/o:redhat:enterprise_linux:10::baseos
Kernel: Linux 6.12.0-55.1.1.el10_0.s390x
Architecture: s390x

------- Comment From Chakravarthi@ibm.com 2025-01-24 04:36 EDT-------
Tried testing the upgrade (9.6 >> 10) without the arg "net.ifnames=0" and observed that network was not coming up on target..

Before upgrade, IP was set with the eth device name enc1

[ 4.530064] cloud-init[961]: ci-info: ++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++
[ 4.530116] cloud-init[961]: ci-info: -------------------------------------------------------------------------+
[ 4.530152] cloud-init[961]: ci-info: | Device | Up | Address | Mask | Scope | Hw-Address |
[ 4.530184] cloud-init[961]: ci-info: -------------------------------------------------------------------------+
[ 4.530215] cloud-init[961]: ci-info: | enc1 | True | 10.20.101.94 | 255.255.255.0 | global | 52:54:00:74:06:ec |
[ 4.530247] cloud-init[961]: ci-info: | enc1 | True | fe80::5054:ff:fe74:6ec/64 | . | link | 52:54:00:74:06:ec |
[ 4.530277] cloud-init[961]: ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | host | . |
[ 4.530308] cloud-init[961]: ci-info: | lo | True | ::1/128 | . | host | . |
[ 4.530339] cloud-init[961]: ci-info: -------------------------------------------------------------------------+

Post upgrade IP was not getting assigned to the guest:

2: enc1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:74:06:ec brd ff:ff:ff:ff:ff:ff
altname enx5254007406ec
inet6 fe80::e8c8:58b:9a22:41c3/64 scope link noprefixroute
valid_lft forever preferred_lft forever

------- Comment From tstaudt@de.ibm.com 2025-02-05 01:53 EDT-------
Hello Petr,

was the previous output helpful or do you require anything else?
Thanks.

------- Comment From Chakravarthi@ibm.com 2025-02-12 02:14 EDT-------
Hello Petr / @thomas ,

Any update on this blocker.. it is blocking our testing.

It would be helpful if there is any fix or workaround for this issue so that we can resume our testing.

------- Comment From Chakravarthi@ibm.com 2025-02-13 09:07 EDT-------
(In reply to comment #20)
> Hi Chakravarthi,
>
> I would like to ensure we all have a common understanding of the desired
> solution.
>
> The description states:
>

Inhibitors not listing for cmdline param net.ifnames=0 during upgrade
> from RHEL 9.6 >> RHEL 10 resulting in the guest not coming up with IP as
> eth0 is no longer supported in RHEL10

Hi,

There are 2 issues here..

1. Inhibitor is not listing if the cmdline param net.ifnames=0 is present as this clearly not supported on RHEL 10.
2. Can't get network on RHEL 10 even after removing the net.ifnames=0 on RHEL 9.6 pre upgrade.

Desired solution would be to fix issue-2 and also add a inhibitor for issue-1 so that upgrade wouldn't go through and break the network of the system.

------- Comment From Chakravarthi@ibm.com 2025-02-13 11:00 EDT-------
Hello @Petr / @Thomas,

Tried a workaround for this issue and it worked (network came up on RHEL 10 post upgrade)

Steps:
1. Create a Rhel 9.6 guest
2. Remove the cmdline param net.ifnames=0 and make sure that network comes up with enc1
3. Migrate the network from sysconfig to NetworkManager – nmcli con migrate /etc/sysconfig/network-scripts/ifcfg-enc1
4. Upgrade the guest to Rhel 10 and verified that network came up.

However, tried to migrate the connection from sysconfig to NetworkManager post-upgrade.. but it fails with the following error –
[root@za93k9-leaap-10-20-101-250 ~]# nmcli con migrate /etc/sysconfig/network-scripts/ifcfg-enc1
Error: unknown connection '/etc/sysconfig/network-scripts/ifcfg-enc1'.
Error: cannot migrate unknown connection(s): '/etc/sysconfig/network-scripts/ifcfg-enc1'.
[root@za93k9-leaap-10-20-101-250 ~]#

So, the desired solution was for this issue would be to migrate the network connections from sysconfig to NetworkManager before upgrade.

either this can be handled as part of upgrade process or raise a inhibitor for this..

Note: this would be a another inhibitor other than the one that was discussed earlier for cmdline param net.ifnames=0

------- Comment From Chakravarthi@ibm.com 2025-02-18 05:07 EDT-------
Tested upgrade with the latest leap build (leapp-0.19.0-1) for rhel 9.6 ? rhel 10.0.
Observed that an inhibitor was raised for legacy network configuration, but not getting any inhibitor for the cmdline param ?net.ifnames=0?
Inhibitor observed for legacy network config

Risk Factor: high (inhibitor)
Title: Legacy network configuration found
Summary: Network configuration files in legacy "ifcfg" format are present.In Red Hat Enterprise Linux 10, support for these files is no longer enabled and the configuration will be ignored. The followw
ing files were found:

• /etc/sysconfig/network-scripts/ifcfg-eth0
  Related links:
• How to migrate the connection from ifcfg to NetworkManager keyfile plugin?: https://access.redhat.com/solutions/7083803
• nmcli(1) manual, describes "connection migrate" sub-command.: https://networkmanager.dev/docs/api/latest/nmcli.html
• nm-settings-ifcfg-rh(5), description of the "ifcfg" format: https://networkmanager.dev/docs/api/latest/nm-settings-ifcfg-rh.html
• nm-settings-keyfile(5), description of the "keyfile" format: https://networkmanager.dev/docs/api/latest/nm-settings-keyfile.html
  Remediation: [hint] Convert the configuration into NetworkManager native "keyfile" format.
  [command] nmcli connection migrate /etc/sysconfig/network-scripts/ifcfg-eth0
  Migrated the legacy network config using ?nmcli con migrate?, but not removed the cmdline param ?net.ifnames=0?

cat /proc/cmdline
root=UUID=671e55ee-3432-4e19-9a5b-d65b446f272e console=tty0 console=ttyS0,115200n8 no_timer_check net.ifnames=0 crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M
For the cmdline param ?net.ifnames=0?, we?re not getting any inhibitor or error –

Reports summary:
Errors: 0
Inhibitors: 0
HIGH severity reports: 1
MEDIUM severity reports: 1
LOW severity reports: 3
INFO severity reports: 2
We expect to get a inhibitor for the cmdline param ?net.ifnames=0?, as this is not supported with Rhel 10.0

------- Comment From Chakravarthi@ibm.com 2025-02-19 22:53 EDT-------
From our testing, we found that if net.ifnames=o kernel parameter is present, then system tries to bring up the network with eth0, which is now not supported by RHEL 10 as now it uses "udev" to assign predictable naming for network interfaces..

Also, I did some googling and found this –

In Red Hat Enterprise Linux (RHEL) 10, using the "net.ifnames=0" kernel parameter is not recommended as it disables the predictable network interface naming scheme which is now the default behavior in RHEL 10; essentially, you cannot use the old "eth0" style naming and should rely on the new, more descriptive names like "ens3" for network interfaces. [1, 2, 3]
Key points about "net.ifnames=0" in RHEL 10: [1, 2, 3]

? Deprecated: Red Hat strongly advises against using "net.ifnames=0" in RHEL 10 as it goes against the new standard of predictable network interface names. [1, 2, 3]
? Older naming scheme: "net.ifnames=0" would revert back to the old "eth0", "eth1" style interface naming which is no longer the default in RHEL 10. [1, 2, 4]
? Potential issues: Using "net.ifnames=0" may lead to compatibility issues with newer applications and configurations that expect the new naming convention. [1, 2, 3]

What to do instead: [1, 2, 3]

? Embrace new naming: Accept the new predictable network interface names provided by RHEL 10 without using "net.ifnames=0".
? Check interface names: Use commands like ip addr show to identify the new names assigned to your network interfaces. [1, 2, 3]

Sources:

[1]?https://www.unixsysadmin.com/rhel-10-resources/[2]?https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/upgrading_from_rhel_7_to_rhel_8/troubleshooting_upgrading-from-rhel-7-to-rhel-8[3]?https://docs.redhat.com/en-us/documentation/red_hat_enterprise_linux/10-beta/pdf/considerations_in_adopting_rhel_10/Red_Hat_Enterprise_Linux-10-beta-Considerations_in_adopting_RHEL_10-en-US.pdf[4]?https://www.ibm.com/support/pages/network-interface-names-rhel-7-sles12-may-get-reordered-system-reboot-ibm-system-cluster-1350-1410

From the above, it says that RedHat no longer support net.ifnames-0 with RHEL 10.

So, we want it to be inhibited and documented..

------- Comment From Chakravarthi@ibm.com 2025-02-20 02:21 EDT-------
Hi Petr,

we did try one scenario and it worked with "net.ifnames=0" present..

steps:
1. create a 9.6 guest with net.ifnames=0 present
2. migrate the eth0 connection using "nmcli con migrate"
3. Upgrade the guest to rhel10 and it passed.

Observed that no inhibitors were raised (as there were no legacy connections at the time of upgrade) & guest upgraded to rhel 10 with eth0 as interface name.

So, inhibitor is not required for net.ifnames=0

Log snippets for the same:

Before upgrade:
====================================================================================
[root@za93ka-leaap-10-20-112-215 yum.repos.d]# cat /proc/cmdline
root=UUID=671e55ee-3432-4e19-9a5b-d65b446f272e console=tty0 console=ttyS0,115200n8 no_timer_check net.ifnames=0 crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M

[root@za93ka-leaap-10-20-112-215 yum.repos.d]# hostnamectl
Static hostname: za93ka-leaap-10-20-112-215
Icon name: computer-vm
Chassis: vm ??
Machine ID: 364954e392ed430eab99dcc55d9b3860
Boot ID: 75eca93a4a1041d2bd8a854d41c1c123
Virtualization: kvm
Operating System: Red Hat Enterprise Linux 9.6 Beta (Plow)
CPE OS Name: cpe:/o:redhat:enterprise_linux:9::baseos
Kernel: Linux 5.14.0-559.el9.s390x
Architecture: s390x

[root@za93ka-leaap-10-20-112-215 yum.repos.d]# ip a
1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:b3:e3:f4 brd ff:ff:ff:ff:ff:ff
altname enc1
inet 10.20.112.215/22 brd 10.20.115.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:feb3:e3f4/64 scope link
valid_lft forever preferred_lft forever
===================================================================================

After upgrade:
====================================================================================
[rundeck@za93ka-leaap-10-20-112-215 ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:b3:e3:f4 brd ff:ff:ff:ff:ff:ff
altname enc1
altname enx525400b3e3f4
inet 10.20.112.215/22 brd 10.20.115.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:feb3:e3f4/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever

[rundeck@za93ka-leaap-10-20-112-215 ~]$ hostnamectl
Static hostname: za93ka-leaap-10-20-112-215
Icon name: computer-vm
Chassis: vm ??
Machine ID: 364954e392ed430eab99dcc55d9b3860
Boot ID: 87e3cc0b704a40ea84a24590292f5737
Virtualization: kvm
Operating System: Red Hat Enterprise Linux 10.0 Beta (Coughlan)
CPE OS Name: cpe:/o:redhat:enterprise_linux:10::baseos
Kernel: Linux 6.12.0-47.el10.s390x
Architecture: s390x

[rundeck@za93ka-leaap-10-20-112-215 ~]$ cat /proc/cmdline
root=UUID=671e55ee-3432-4e19-9a5b-d65b446f272e console=tty0 console=ttyS0,115200n8 no_timer_check net.ifnames=0 crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M
====================================================================================

Hi rhn-support-mkluson, 

Thanks for the clarifications, we will finally work on making nmcli con migrate to migrate route and rule files, how could we show better our involvement in this. Should I change the component to NetworkManager? 

Thanks

Reassigning to leapp to assess this bug based on previous comment. 

The rule files are handled by a dispatcher script contained in the "NetworkManager-dispatcher-routing-rules" package; the NetworkManager daemon doesn't have any understanding of those files.

So, handling the migration of those files during "nmcli connection migrate" would require some implementation effort; furthermore those files can contain table names, while NM only supports table numbers and so we would need to handle the translation by looking up in the iproute2 mapping files.

As NetworkManager support for ifcfg files is deprecated and is going to be removed in future versions, I don't think we want to add new code to handle legacy files.

Maybe this should be handled by Leapp by either:

 - blocking the upgrade and asking the user to: first migrate the connections with "nmcli connection migrate", then convert the rules manually

 - implementing the conversion

As NetworkManager has native support for policy based routing I believe "nmcli con migrate" should also be able to bring in the routes and rules from the route-<interface> and rule-<interface> files.

This was brought to my attention in a support case where a customer performed a LEAP upgrade from RHEL 7 to RHEL 8. Then did a LEAP upgrade from RHEL 8 to RHEL 9. During the upgrade to RHEL 9 the customer was instructed to run  "nmcli con migrate" as the initscripts are no longer available in RHEL 9. Upon booting into RHEL 9 the customer found that none of their routes and rules were present and as such had broken connectivity.