Uploaded image for project: 'Arquillian'
  1. Arquillian
  2. ARQ-1282

TestNG @DataProvider is fully invoked for each record

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 1.0.3.Final
    • JBoss AS Containers
    • None
    • Hide

      TestCase Class

      DataProviderTest.java
      import org.jboss.arquillian.container.test.api.Deployment;
import org.jboss.arquillian.container.test.api.OverProtocol;
import org.jboss.arquillian.testng.Arquillian;
import org.jboss.shrinkwrap.api.Archive;
import org.jboss.shrinkwrap.api.ShrinkWrap;
import org.jboss.shrinkwrap.api.asset.EmptyAsset;
import org.jboss.shrinkwrap.api.spec.WebArchive;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

import static org.testng.Assert.*;

public class DataProviderTest extends Arquillian {

    @Deployment
    @OverProtocol( "Servlet 3.0" )
    public static Archive<?> getDeployment() {
        return ShrinkWrap.create( WebArchive.class ).addAsWebInfResource( EmptyAsset.INSTANCE, "beans.xml" );
    }

    @Test( dataProvider = "sumProvider" )
    public void testSum( int a, int b, int expectedSum ) {

        // execute test
        int result = a + b;

        // log it and watch wrong behaviour
        // HERE IS A BUG !!!
        // Look into server log to NUMBER of logged events.
        System.out.println( String.format( "Execution of sum: %1$d + %2$d = %3$d", a, b, result ) );

        // verify result
        assertEquals( result, expectedSum );
    }

    @DataProvider
    public Object[][] sumProvider() {
        return new Object[][]{
                new Object[]{ 0, 0, 0 },
                new Object[]{ 1, 1, 2 },
                new Object[]{ 1, 5, 6 },
                new Object[]{ 5, 1, 6 }
        };
    }

}
      Show
      TestCase Class DataProviderTest.java import org.jboss.arquillian.container.test.api.Deployment; import org.jboss.arquillian.container.test.api.OverProtocol; import org.jboss.arquillian.testng.Arquillian; import org.jboss.shrinkwrap.api.Archive; import org.jboss.shrinkwrap.api.ShrinkWrap; import org.jboss.shrinkwrap.api.asset.EmptyAsset; import org.jboss.shrinkwrap.api.spec.WebArchive; import org.testng.annotations.DataProvider; import org.testng.annotations.Test; import static org.testng.Assert.*; public class DataProviderTest extends Arquillian { @Deployment @OverProtocol( "Servlet 3.0" ) public static Archive<?> getDeployment() { return ShrinkWrap.create( WebArchive.class ).addAsWebInfResource( EmptyAsset.INSTANCE, "beans.xml" ); } @Test( dataProvider = "sumProvider" ) public void testSum( int a, int b, int expectedSum ) { // execute test int result = a + b; // log it and watch wrong behaviour // HERE IS A BUG !!! // Look into server log to NUMBER of logged events. System .out.println( String .format( "Execution of sum: %1$d + %2$d = %3$d" , a, b, result ) ); // verify result assertEquals( result, expectedSum ); } @DataProvider public Object [][] sumProvider() { return new Object [][]{ new Object []{ 0, 0, 0 }, new Object []{ 1, 1, 2 }, new Object []{ 1, 5, 6 }, new Object []{ 5, 1, 6 } }; } }

      Expected behavior

      TestNG defines @DataProvider annotation to allow single method to perform multiple test cases. Provider method annotated with @DataProvider returns an array of arrays, each containing one test case. These parameters are given to the target method as input parameters. By the definition, the total count of performed test by single method is equal to number of test cases in related data provider.

      Actual behavior

      Using TestNG under Arquillian framework makes this functionality buggy. Although the client (maven, IDE, etc.) thinks, that the total number of invoked test is correct, it is not. When we take a look into server's log, we can see that for each single test case it invoked full set of all test cases. It means, that in the end the amount of performed tests is equal to expected count squared. Such behaviour is not only slow but also in some cases it doesn't work and it makes tests to fail.

      Example

      @DataProvider
public Object[][] sumProvider() {
    return new Object[][]{
        new Object[]{ 0, 0, 0 },
        new Object[]{ 1, 1, 2 },
        new Object[]{ 1, 5, 6 },
        new Object[]{ 5, 1, 6 }
    };
}

      Expected output in server log

      Execution of sum: 0 + 0 = 0
      Execution of sum: 1 + 1 = 2
      Execution of sum: 1 + 5 = 6
      Execution of sum: 5 + 1 = 6

      Actual output

      Execution of sum: 0 + 0 = 0
      Execution of sum: 1 + 1 = 2
      Execution of sum: 1 + 5 = 6
      Execution of sum: 5 + 1 = 6
      Execution of sum: 0 + 0 = 0
      Execution of sum: 1 + 1 = 2
      Execution of sum: 1 + 5 = 6
      Execution of sum: 5 + 1 = 6
      Execution of sum: 0 + 0 = 0
      Execution of sum: 1 + 1 = 2
      Execution of sum: 1 + 5 = 6
      Execution of sum: 5 + 1 = 6
      Execution of sum: 0 + 0 = 0
      Execution of sum: 1 + 1 = 2
      Execution of sum: 1 + 5 = 6
      Execution of sum: 5 + 1 = 6

        1. ArquillianDataProvider.java
          1 kB
        2. FlightServiceTest.java
          9 kB

              bartosz-1 Bartosz Majsak
              karel.cemus Karel Cemus (Inactive)
              Votes:
              13 Vote for this issue
              Watchers:
              14 Start watching this issue

                Created:
                Updated: