Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-5527

Second policy to create SR-IOV VFs failed

    • nmstate-2.2.21-1.el9
    • None
    • None
    • 1
    • rhel-sst-network-management
    • ssg_networking
    • 26
    • 1
    • False
    • No
    • NMT - RHEL 8.10/9.4 DTM 14
    • Hide

      Given a system administrator configuring an OpenShift cluster with two Intel interfaces (ens1f0 and ens1f1) configured for SR-IOV,

      When they apply an Nmstate policy to create 5 VFs on the ens1f0 interface and another Nmstate policy to create 5 VFs on the ens1f1 interface,

      Then, both Nmstate policies are successfully applied without errors, and 5 VFs are created on each interface and NetworkManager connections, modified by Nmstate, do not deactivate due to DHCP timeouts, as ipv4.dhcp-timeout and ipv6.dhcp-timeout are set to i32::MAX

      Definition of Done:

      • The implementation meets the acceptance criteria
      • The integration tests are written and pass
      • The code is part of a downstream build attached to an errata
      Show
      Given a system administrator configuring an OpenShift cluster with two Intel interfaces (ens1f0 and ens1f1) configured for SR-IOV, When they apply an Nmstate policy to create 5 VFs on the ens1f0 interface and another Nmstate policy to create 5 VFs on the ens1f1 interface, Then, both Nmstate policies are successfully applied without errors, and 5 VFs are created on each interface and NetworkManager connections, modified by Nmstate, do not deactivate due to DHCP timeouts, as ipv4.dhcp-timeout and ipv6.dhcp-timeout are set to i32::MAX Definition of Done: The implementation meets the acceptance criteria The integration tests are written and pass The code is part of a downstream build attached to an errata
    • Pass
    • None
    • None

      Description of problem:

      Second policy failed after ~10 min
       progressingNmstateError: VerificationError: Verification failure: ens1f1.interface.ethernet.sr-iov.total-vfs desire '5', current '0'

      Version-Release number of selected component (if applicable):

      Server Version: 4.13.4
      kubernetes-nmstate-operator.4.13.0-202306290616

      How reproducible:

      70%

      Steps to Reproduce:

      1.Apply nmstate policy to create VFs  on Intel interface ens1f0 - Applied, VFs created
      
          name: configurevfs
          desiredState:
            interfaces:
            - ethernet:
                sr-iov:
                  total-vfs: 5
              name: ens1f0
              state: up
              type: ethernet
          nodeSelector:
            node-role.kubernetes.io/worker: ""
      2.Appy anotoher nmstate policy to created VFs on Intel interface ens1f1
      
          name: configurevfs5
          desiredState:
            interfaces:
            - ethernet:
                sr-iov:
                  total-vfs: 5
              name: ens1f1
              state: up
              type: ethernet
          nodeSelector:
            node-role.kubernetes.io/worker: ""
      
      
      
      

      Actual results:

      ]$ oc get nnce 
      NAME                     STATUS      REASON
      worker-0.configurevfs    Available   SuccessfullyConfigured
      worker-0.configurevfs5   Failing     FailedToConfigure
      worker-1.configurevfs    Available   SuccessfullyConfigured
      worker-1.configurevfs5   Aborted     ConfigurationAborted

      Expected results:

      ]$ oc get nnce 
      NAME                     STATUS      REASON
      worker-0.configurevfs    Available   SuccessfullyConfigured
      worker-0.configurevfs5   Available   SuccessfullyConfigured
      worker-1.configurevfs    Available   SuccessfullyConfigured
      worker-1.configurevfs5   Available   SuccessfullyConfigured

      Additional info:

      VFs created on ens1f1
      oc describe nnce worker-0.configurevfs5
      http://pastebin.test.redhat.com/1104566
      
      

       
      sh-5.1# nmcli con
      NAME UUID TYPE DEVICE
      Wired Connection 365804f7-e72c-441a-935b-4ed113f8825a ethernet ens5f0
      Wired Connection 365804f7-e72c-441a-935b-4ed113f8825a ethernet ens5f1
      Wired Connection 365804f7-e72c-441a-935b-4ed113f8825a ethernet ens8f0np0
      Wired Connection 365804f7-e72c-441a-935b-4ed113f8825a ethernet ens8f1np1
      ovs-if-br-ex 715bcca2-131a-4cad-809c-933c4b97cd46 ovs-interface br-ex
      lo 4b6cd6e2-67a4-4be0-9736-52ee51c96d44 loopback lo
      br-ex 9f5cd37a-6c59-4df4-a270-83ce423d19fd ovs-bridge br-ex
      ens1f0 d3cc5f99-822e-4dcb-b88c-3ad3e5458d35 ethernet ens1f0
      ens1f1 e2390174-50ec-4062-891b-2ef32e5f8d89 ethernet ens1f1
      ovs-if-phys0 671d035c-34db-4ea3-ae9a-dace10c2310a ethernet eno1
      ovs-port-br-ex 60c9be79-d625-47dd-9ce4-a7c6a403cd50 ovs-port br-ex
      ovs-port-phys0 71bca81e-e100-4a92-89c2-fe838ca943fc ovs-port eno1
      STATIC c7711bad-7042-4ac9-90f9-fa6dccec5565 ethernet –
      STATIC cf4de431-f940-4893-a1c3-3054e5580dfb ethernet –
      STATIC ca2de388-ce50-4d2e-95ac-4543e1bf9436 ethernet –
      STATIC 4ce4c886-b2c3-4fa6-b23b-448153d6baaa ethernet –
      Wired Connection 4d1c54a5-7295-4940-952d-fc05ec333a30 ethernet –

      journalctl:
      http://pastebin.test.redhat.com/1104567

      nns:

      • accept-all-mac-addresses: false
        description: Create VFs
        ethernet:
        auto-negotiation: false
        duplex: full
        speed: 25000
        sr-iov:
        total-vfs: 5
        vfs:
      • id: 0
        mac-address: "00:00:00:00:00:00"
        max-tx-rate: 0
        min-tx-rate: 0
        qos: 0
        spoof-check: true
        trust: false
        vlan-id: 0
      • id: 1
        mac-address: "00:00:00:00:00:00"
        max-tx-rate: 0
        min-tx-rate: 0
        qos: 0
        spoof-check: true
        trust: false
        vlan-id: 0
      • id: 2
        mac-address: "00:00:00:00:00:00"
        max-tx-rate: 0
        min-tx-rate: 0
        qos: 0
        spoof-check: true
        trust: false
        vlan-id: 0
      • id: 3
        mac-address: "00:00:00:00:00:00"
        max-tx-rate: 0
        min-tx-rate: 0
        qos: 0
        spoof-check: true
        trust: false
        vlan-id: 0
      • id: 4
        mac-address: "00:00:00:00:00:00"
        max-tx-rate: 0
        min-tx-rate: 0
        qos: 0
        spoof-check: true
        trust: false
        vlan-id: 0
        ethtool:
        coalesce:
        adaptive-rx: true
        adaptive-tx: true
        rx-usecs: 50
        rx-usecs-high: 0
        tx-frames-irq: 256
        tx-usecs: 50
        tx-usecs-high: 0
        feature:
        highdma: true
        hw-tc-offload: false
        l2-fwd-offload: false
        rx-checksum: true
        rx-gro: true
        rx-gro-list: false
        rx-hashing: true
        rx-ntuple-filter: true
        rx-udp-gro-forwarding: false
        rx-udp_tunnel-port-offload: true
        rx-vlan-hw-parse: true
        tx-checksum-ip-generic: true
        tx-checksum-sctp: true
        tx-generic-segmentation: true
        tx-gre-csum-segmentation: true
        tx-gre-segmentation: true
        tx-gso-partial: true
        tx-ipxip4-segmentation: true
        tx-ipxip6-segmentation: true
        tx-nocache-copy: false
        tx-tcp-ecn-segmentation: true
        tx-tcp-mangleid-segmentation: false
        tx-tcp-segmentation: true
        tx-tcp6-segmentation: true
        tx-udp-segmentation: true
        tx-udp_tnl-csum-segmentation: true
        tx-udp_tnl-segmentation: true
        tx-vlan-hw-insert: true
        pause:
        autoneg: false
        rx: false
        tx: false
        ring:
        rx: 512
        rx-max: 4096
        tx: 512
        tx-max: 4096
        ipv4:
        dhcp: false
        enabled: false
        ipv6:
        autoconf: false
        dhcp: false
        enabled: false
        lldp:
        enabled: false
        mac-address: 40:A6:B7:71:B2:A1
        max-mtu: 9702
        min-mtu: 68
        mtu: 1500
        name: ens1f1
        state: up
        type: ethernet
        wait-ip: any

              rh-ee-mshi1 Mingyu Shi
              rhn-cnf-elevin Evgeny Levin
              Gris Ge Gris Ge
              Mingyu Shi Mingyu Shi
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

                Created:
                Updated:
                Resolved: