Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-12852

BZ#2215292 several tobiko failures regarding subnet operations on neutron

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Undefined Undefined
    • None
    • rhos-17.1.z
    • openstack-neutron
    • False
    • False
    • None

      Description of problem:
      Filed as Bug Deputy.
      I'm not 100% sure if this failure is due to tobiko or to neutron. More investigation would be necessary. Hopefully these inputs are useful.

      In the gate DFG-network-neutron-17.1_director-rhel-virthost-3cont_2comp-ml2ovs-ipv4-vxlan- tobiko-tempest-nossl-ovsfirewall we find that in both
      tobiko_faults-neutron_1_02_create_neutron_resources_neutron and tobiko_faults-neutron_1_06_verify_neutron_resources_neutron this tests are failing without being able to reach the final server.

      Test_2_ping_vlan_port_after_restart
      Test_1_ping_vlan_port
      Test_0_vlan_ip_addresses

      Fail with:

      Traceback (most recent call last):
      File "/home/stack/src/x/tobiko/tobiko/tests/scenario/neutron/test_trunk.py", line 90, in test_2_ping_vlan_port_after_restart
      self.stack.assert_vlan_is_reachable()
      File "/home/stack/src/x/tobiko/tobiko/openstack/stacks/_vlan.py", line 103, in assert_vlan_is_reachable
      ping.assert_reachable_hosts(fixed_ips,
      File "/home/stack/src/x/tobiko/tobiko/shell/ping/_assert.py", line 27, in assert_reachable_hosts
      _ping.wait_for_ping_hosts(hosts, **params)
      File "/home/stack/src/x/tobiko/tobiko/shell/ping/_ping.py", line 100, in wait_for_ping_hosts
      raise _exception.UnreachableHostsException(
      tobiko.shell.ping._exception.UnreachableHostsException: timeout of 900.0 seconds expired after 904.4510264396667 seconds waiting for host(s) Selection([IPAddress('10.100.4.30'), IPAddress('fc00::4:f816:3eff:feb5:46e4')]) to return ping messages

      Looking at the logs of the controllers for this gate [0] I found:
      In server.log.1.gz, many logs with the following 404 error:

      2023-06-14 03:03:50.900 16 INFO neutron.services.segments.plugin [-] Segment 81a4475f-9e3b-4aae-843b-527dd359530e resource provider not found; error: Placement Client Error (4xx): {"errors": [

      {"status": 404, "title": "Not Found", "detail": "The resource could not be found.\n\n No resource provider with uuid 81a4475f-9e3b-4aae-843b-527dd359530e found for delete ", "request_id": "req-9338d486-ebd4-4426-9c87-99815fcd44aa"}

      ]}

      And also many "subnet is already being modified" errors:
      2023-06-14 03:03:24.873 20 ERROR oslo_messaging.rpc.server raise exception
      2023-06-14 03:03:24.873 20 ERROR oslo_messaging.rpc.server neutron_lib.exceptions.SubnetInUse: Unable to complete operation on subnet 49eb8a84-9d6e-44ed-b9e5-77a57d8a955d: This subnet is being modified by another concurrent operation.

      Version-Release number of selected component (if applicable):
      17.1

      How reproducible:
      Not sure, we should check the next build of DFG-network-neutron-17.1_director-rhel-virthost-3cont_2comp-ml2ovs-ipv4-vxlan-tobiko-tempest-nossl-ovsfirewall [1]

      [0] http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/DFG-network-neutron-17.1_director-rhel-virthost-3cont_2comp-ml2ovs-ipv4-vxlan-tobiko-tempest-nossl-ovsfirewall/33/

      [1] https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/job/DFG-network-neutron-17.1_director-rhel-virthost-3cont_2comp-ml2ovs-ipv4-vxlan-tobiko-tempest-nossl-ovsfirewall/

      Actual results:
      Test_2_ping_vlan_port_after_restart
      Test_1_ping_vlan_port
      Test_0_vlan_ip_addresses
      fails in both tobiko_faults-neutron_1_02_create_neutron_resources_neutron and tobiko_faults-neutron_1_06_verify_neutron_resources_neutron
      Expected results:
      Test are successful

              froyo@redhat.com Fernando Royo
              jira-bugzilla-migration RH Bugzilla Integration
              Eran Kuris Eran Kuris
              rhos-dfg-networking-squad-neutron
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: