Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-20339

Error with `remove_config` and nmstate provider with test_remove_con branch

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • os-net-config
    • None
    • Important

      Hit this error:

      2025-09-26 04:43:29.343 ERROR os_net_config.config_provider nmstate: ***Failed to configure network_config ***
      Verification failure: ens1f0.interface.ethernet.sr-iov.drivers-autoprobe desire 'true', current 'false'
      Traceback (most recent call last):
        File "/usr/lib/python3.9/site-packages/os_net_config/cli.py", line 593, in config_provider
          pf_files_changed = provider.apply(cleanup=False,
        File "/usr/lib/python3.9/site-packages/os_net_config/impl_nmstate.py", line 2845, in apply
          self.rollback_to_initial_settings()
        File "/usr/lib/python3.9/site-packages/os_net_config/impl_nmstate.py", line 425, in rollback_to_initial_settings
          netapplier.apply(diff_state, verify_change=True)
        File "/usr/lib/python3.9/site-packages/libnmstate/netapplier.py", line 32, in apply
          return apply_net_state(
        File "/usr/lib/python3.9/site-packages/libnmstate/clib_wrapper.py", line 133, in apply_net_state
          raise map_error(err_kind, err_msg)
      libnmstate.error.NmstateVerificationError: Verification failure: ens1f0.interface.ethernet.sr-iov.drivers-autoprobe desire 'true', current 'false'
      

      When testing the support for support NIC partitioning for the interface that was used for baremetal provisioning for nmstate provider (see OSPRH-20281)

      NetworkManager version:

      [cloud-admin@compute-1 ~]$ rpm -qi NetworkManager
      Name        : NetworkManager
      Epoch       : 1
      Version     : 1.46.0
      Release     : 35.el9_4
      Architecture: x86_64
      Install Date: Fri 26 Sep 2025 04:32:30 AM EDT
      Group       : System Environment/Base
      Size        : 6593397
      License     : GPL-2.0-or-later AND LGPL-2.1-or-later
      Signature   : RSA/SHA256, Fri 12 Sep 2025 09:01:11 AM EDT, Key ID 199e2f91fd431d51
      Source RPM  : NetworkManager-1.46.0-35.el9_4.src.rpm
      Build Date  : Wed 10 Sep 2025 03:20:46 PM EDT
      Build Host  : x86-64-04.build.eng.rdu2.redhat.com
      Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
      Vendor      : Red Hat, Inc.
      URL         : https://networkmanager.dev/
      Summary     : Network connection manager and user applications
      Description :
      NetworkManager is a system service that manages network interfaces and
      connections based on user or automatic configuration. It supports
      Ethernet, Bridge, Bond, VLAN, Team, InfiniBand, Wi-Fi, mobile broadband
      (WWAN), PPPoE and other devices, and supports a variety of different VPN
      services.
      

      To Reproduce:

      1. Apply this OpenStackDataPlaneNodeSet CR (notice that remove_config is used for ens1f1):
        apiVersion: dataplane.openstack.org/v1beta1
        kind: OpenStackDataPlaneNodeSet
        metadata:
          name: openstack-edpm
          namespace: openstack
        spec:
          baremetalSetTemplate:
            ctlplaneInterface: ens1f1
          ...
          nodeTemplate:
            ansible:
                ...
                edpm_bootstrap_command: |-
                  # Replace os-net-config
                  cd /tmp/
                  git clone https://github.com/karthiksundaravel/os-net-config_nmstate.git -b test_remove_con
                  cd os-net-config_nmstate/
                  rm -fr /usr/lib/python3.9/site-packages/os_net_config
                  mv os_net_config /usr/lib/python3.9/site-packages/
               ...
                edpm_network_config_nmstate: true
                edpm_network_config_nonconfigured_cleanup: false
                edpm_network_config_os_net_config_mappings:
                  edpm-compute-0:
                    nic1: 'ec:2a:72:40:b4:28'  # eno8303
                    nic2: '6c:fe:54:3f:87:00'  # ens1f0
                    nic3: '6c:fe:54:3f:87:01'  # ens1f1
                edpm_network_config_template: |
                  ---
                  {% set mtu_list = [ctlplane_mtu] %}
                  {% for network in nodeset_networks %}
                  {{ mtu_list.append(lookup('vars', networks_lower[network] ~ '_mtu')) }}
                  {%- endfor %}
                  {% set min_viable_mtu = mtu_list | max %}
                  remove_config:
                  - remove_type: interface
                    remove_name: nic3
                  network_config:
                  - type: interface
                    name: nic1
                    use_dhcp: false
                  - type: sriov_pf
                    name: nic2
                    mtu: 9000
                    numvfs: 5
                    use_dhcp: false
                    defroute: false
                    nm_controlled: true
                    hotplug: true
                    drivers_autoprobe: false
                  - type: linux_bond
                    name: ctlplane_bond
                    use_dhcp: false
                    bonding_options: "mode=active-backup"
                    dns_servers: {{ ctlplane_dns_nameservers }}
                    members:
                    - type: sriov_vf
                      device: nic2
                      vfid: 0
                    addresses:
                    - ip_netmask: {{ ctlplane_ip }}/{{ ctlplane_cidr }}
                    routes:
                    - default: true
                      next_hop: {{ ctlplane_gateway_ip }}
                  - type: sriov_pf
                    name: nic3
                    mtu: 9000
                    numvfs: 5
                    use_dhcp: false
                    defroute: false
                    nm_controlled: true
                    hotplug: true
                    drivers_autoprobe: false
                  - type: linux_bond
                    name: bond_api
                    use_dhcp: false
                    bonding_options: "mode=active-backup"
                    dns_servers: {{ ctlplane_dns_nameservers }}
                    members:
                    - type: sriov_vf
                      device: nic3
                      vfid: 0
                      vlan_id: {{ lookup('vars', networks_lower['internalapi'] ~ '_vlan_id') }}
                    addresses:
                    - ip_netmask: {{ lookup('vars', networks_lower['internalapi'] ~ '_ip') }}/{{ lookup('vars', networks_lower['internalapi'] ~ '_cidr') }}
                  - type: linux_bond
                    name: storage_bond
                    use_dhcp: false
                    bonding_options: "mode=active-backup"
                    dns_servers: {{ ctlplane_dns_nameservers }}
                    members:
                    - type: sriov_vf
                      device: nic3
                      vfid: 1
                      vlan_id: {{ lookup('vars', networks_lower['storage'] ~ '_vlan_id') }}
                    addresses:
                    - ip_netmask: {{ lookup('vars', networks_lower['storage'] ~ '_ip') }}/{{ lookup('vars', networks_lower['storage'] ~ '_cidr') }}
                  - type: ovs_user_bridge
                    name: br-link0
                    use_dhcp: false
                    mtu: 9000
                    ovs_extra: "set port br-link0 tag={{ lookup('vars', networks_lower['tenant'] ~ '_vlan_id') }}"
                    addresses:
                    - ip_netmask: {{ lookup('vars', networks_lower['tenant'] ~ '_ip') }}/{{ lookup('vars', networks_lower['tenant'] ~ '_cidr') }}
                    members:
                    - type: ovs_dpdk_port
                      rx_queue: 1
                      name: dpdk0
                      members:
                      - type: sriov_vf
                        device: nic3
                        vfid: 2
          ...
        
      2. Apply the corresponding OpenStackDataPlaneDeployment
      3. You'll see error pods like these:
        [zuul@controller-0 ~]$ oc get pod |grep configure-network                                                                                                                                                                                                                                                                     
        configure-network-edpm-deployment-openstack-edpm-2rd8v            0/1     Error       0          13m                                                                                                                                                                                                                          
        configure-network-edpm-deployment-openstack-edpm-7rlll            0/1     Error       0          2m17s                                                                                                                                                                                                                        
        configure-network-edpm-deployment-openstack-edpm-87tb5            0/1     Error       0          2m43s                                                                                                                                                                                                                        
        configure-network-edpm-deployment-openstack-edpm-fzlqv            0/1     Error       0          89s             
        

      Expected behavior

      • configure-network run without errors

              ksundara@redhat.com Karthik Sundaravel
              rdiazcam@redhat.com Ricardo Diaz Campos
              rhos-dfg-nfv
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: