Uploaded image for project: 'AMQ Streams'
  1. AMQ Streams
  2. ENTMQST-4870

[QE] Reported number of executed tests in strimzi-pull-request-job does not match real number of executed tests.

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Done
    • Icon: Minor Minor
    • None
    • None
    • None
    • False
    • None
    • False

      Description

      Due to known surefire maven problem, we may encounter situations (when running Test Classes with several tests) that due to Surefire report containing wrong totals on number of tests this number is later propagated to report in our job (strimzi-pull-request-job).

      Example and how to reproduce

      for example when running : @strimzi-ci run tests --testcase=TopicST expected to execute ~8 tests we will end with following report (while 8 test are in reality executed) 
      **

      TEST_PROFILE: all
      EXCLUDED_GROUPS: loadbalancer,nodeport,olm
      TEST_CASE: TopicST
      TOTAL: 1
      PASS: 1
      FAIL: 0
      SKIP: 0
      BUILD_NUMBER: 88
      OCP_VERSION: 4.13
      BUILD_IMAGES: false
      FIPS_ENABLED: false
      TEST_COUNT_RUNNING_IN_PARALLEL: 5

      can be seen in https://github.com/strimzi/strimzi-kafka-operator/pull/8475  

      Probably linked to: 

      Already discussed in https://issues.apache.org/jira/browse/SUREFIRE-1627 

              jstejska@redhat.com Jakub Stejskal
              hzrncik_kafka_test Henrich Zrncik
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: