Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-14950

[CI][Neutron] Nested routers test failing on network component DVR job (blocks OSP18 promotion)

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Blocker Blocker
    • None
    • None
    • internal
    • 2
    • False
    • Hide

      None

      Show
      None
    • False
    • ?
    • None
    • Critical

      At first sight CI results looked alarming as if bug is in new nested SNAT feature introduced, and affecting only DVR setups
      (which were not checked with feature as far as I know).

      The 3 last `component-network-edpm-rhel9-rhoso18.0-crc-non-dvr` job runs failing since March 20 [1] on feature related test while passing on latest 22 March run of parent job `component-network-edpm-rhel9-rhoso18.0-crc` [2].

      After checking the CI jobs definition [3], issue looks CI configuration related - the component DVR job inherits part of job configuration, causing feature to be disabled (by default) in OVN, but tests to be enabled by tempest configuration.

      Testing fix to properly configure and test DVR setups using network component job with new feature.

      Side note:
      This fix is to verify feature with DVR and unblock promotion quickly, but whitebox version of same test also checks ovn configuration automatically within tempest and checks nested SNAT connectivity passes/fails accordingly, should be used instead when test available - sparing such CI future maintenance with tempest configuration.  

       

      [1] 

      https://sf.apps.int.gpc.ocp-hub.prod.psi.redhat.com/zuul/t/components-integration/builds?job_name=component-network-edpm-rhel9-rhoso18.0-crc-non-dvr&project=ci-framework/ci-framework-jobs 

      https://sf.apps.int.gpc.ocp-hub.prod.psi.redhat.com/logs/191/components-integration/191574968f3a4bfaafb4fa6cfd55f01e/controller/ci-framework-data/tests/test_operator/tempest-tests-tempest-workflow-step-00-multithread/stestr_results.html 

      https://sf.apps.int.gpc.ocp-hub.prod.psi.redhat.com/logs/dc8/components-integration/dc8b6ce1398d47429b7644a9c19f30e7/controller/ci-framework-data/tests/test_operator/tempest-tests-tempest-workflow-step-00-multithread/stestr_results.html 

      https://sf.apps.int.gpc.ocp-hub.prod.psi.redhat.com/logs/cde/components-integration/cdeed1f06023443abe0ca1191978a8f1/controller/ci-framework-data/tests/test_operator/tempest-tests-tempest-workflow-step-00-multithread/stestr_results.html 

       

      Traceback (most recent call last):
        File "/var/lib/tempest/external-plugins/.venv/lib64/python3.9/site-packages/tempest/common/utils/__init__.py", line 84, in wrapper
          return func(*func_args, **func_kwargs)
        File "/var/lib/tempest/external-plugins/.venv/lib64/python3.9/site-packages/neutron_tempest_plugin/scenario/test_floatingip.py", line 260, in test_nested_snat_external_ip
          self._test_nested_snat_external_ip(feature_enabled_bool)
        File "/var/lib/tempest/external-plugins/.venv/lib64/python3.9/site-packages/neutron_tempest_plugin/scenario/test_floatingip.py", line 249, in _test_nested_snat_external_ip
          self.check_remote_connectivity(ssh_client,
        File "/var/lib/tempest/external-plugins/.venv/lib64/python3.9/site-packages/neutron_tempest_plugin/scenario/base.py", line 482, in check_remote_connectivity
          self.assertTrue(self._check_remote_connectivity(
        File "/usr/lib64/python3.9/unittest/case.py", line 688, in assertTrue
          raise self.failureException(msg)
      AssertionError: False is not true

      [2]

      https://sf.apps.int.gpc.ocp-hub.prod.psi.redhat.com/zuul/t/components-integration/builds?job_name=component-network-edpm-rhel9-rhoso18.0-crc&project=ci-framework/ci-framework-jobs 

      https://sf.apps.int.gpc.ocp-hub.prod.psi.redhat.com/logs/d9c/components-integration/d9c8c42de0f4472d99e80173b0faf06c/controller/ci-framework-data/tests/test_operator/tempest-tests-tempest-workflow-step-00-multithread/stestr_results.html 

      [3]

      https://gitlab.cee.redhat.com/ci-framework/ci-framework-jobs/-/blob/main/zuul.d/network-component-jobs-rhoso-18-rhel9.yaml#L198 

              rhn-support-mblue Maor Blaustein
              rhn-support-mblue Maor Blaustein
              rhos-dfg-networking-squad-neutron
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: