Internal QA info: http://intranet.corp.redhat.com/ic/intranet/TestingEmbeddedConsoleWithAS.html

JIRA issue: https://jira.jboss.org/jira/browse/JBQA-1938

Changelog: http://fisheye.jboss.org/changelog/EMBJOPR/

Hudson job: http://hudson.qa.jboss.com/hudson/job/embjopr-jsfunit/

Embedded Jopr project JIRA: https://jira.jboss.org/jira/browse/EMBJOPR

Hudson job: http://hudson.qa.jboss.com/hudson/job/embjopr/

Current versions

We're testing with (versions [and reasons for that version to be chosen]):

EmbJopr: 1.1.1-SNAPSHOT (? Shelly will correct.)

JBoss AS 4: jboss-eap-4.3.0_CP03

JBoss AS 5: jboss-5.1.0-Beta1 (Unreleased)

JDK: SUN 1.5.0 - latest update (currently 17)

JSFUnit: 1.1.0.GA-SNAPSHOT

Running Embedded Jopr tests

1. Download and unzip JBoss AS / EAP distribution.
2. Set the JBOSS_HOME property in maven's setting.xml to that JBoss directory.
3. Checkout embjopr project from SVN.
4. Set the root project and jbas4 + jbas5 modules to skip tests in the IDE of choice, or use -Dmaven.skip.test=true in later steps 3 and 4. This is because Seam tests fail.
   
5. Build the root project recursively (this makes Maven downloads dependencies)
6. Build again the jbas4 / jbas5 module
   
7. Build the jsfunit module with the jboss4.2 or jboss5x profile

The last step includes launching JBoss AS in the JBOSS_HOME directory using the Maven Cargo plugin. Do not deploy the resulting war manually on that server, otherwise Cargo will behave quite weirdly (error parsing ServletRedirector.jsfunit, 404 Not found etc). If getting exceptions like this, make sure there's not the war deployed accidentally (in my case, it happened when I invoked non-standard maven goal; thanks to Farah Juma for pointing this out).

See also: JSFUnitWithMaven<br />

Running tests on Hudson: (TBD)

Example settings.xml

<?xml version="1.0" encoding="UTF-8"?>
<settings>
  <profiles>
    <profile>
      <id>jboss4.2</id>
      <properties>
        <JBOSS_HOME>/home/ondra/work/JOPRembedded/jboss-eap-4.3/jboss-as</JBOSS_HOME>
        <JAVA_HOME>/home/ondra/sw/java/jdk/jdk1.5.0_17</JAVA_HOME>
      </properties>
    </profile>     
    <profile>
      <id>jboss5x</id>
      <properties>
        <JBOSS_HOME>/home/ondra/work/JOPRembedded/jboss-5.0.0.GA</JBOSS_HOME>
        <JAVA_HOME>/home/ondra/sw/java/jdk/jdk1.5.0_17</JAVA_HOME>
      </properties>
    </profile>
  </profiles>
</settings>

Running a single / selected test class

mvn -Pjboss5x -Dtest=org.jboss.jopr.jsfunit.as5.DatasourceTest

In fact, test property takes a mask as it's value, so you can use asterisk to filter a group of tests:

mvn -Pjboss5x -Dtest=Datasource*Test

See http://maven.apache.org/plugins/maven-surefire-plugin/examples/single-test.html.

If you need to run single method, there's no (known) way to do it.

AS 5 issues when running JSFUnit tests with Maven Cargo plugin

• Getting OutOfMemoryException: PermGen Error? See Cargo Maven plugin - raising PermGen for JBoss AS 5.
• Tests running too slowly? See Cargo Maven plugin - JBoss AS 5 logging configuration.
• Can we use light JBoss configurations for testing some EmbJopr modules? JBoss 5.x Tuning/Slimming

Tests architecture

The tests are designed to cover both AS 4 (written using EAP 4.3.0) and AS 5 using the same code base, and covering the differences by overriding base classes.

Package hierarchy

• org.jboss.jopr.jsfunit
• org.jboss.jopr.jsfunit.as4
• org.jboss.jopr.jsfunit.as5

The org.jboss.jopr.jsfunit package contains the base class EmbjoprTestCase with supporting methods which deal with common tasks of EmbJopr manipulation.

Other classes in this package, like DatasourceTestBase, inherit from the EmbjoprTestCase class. Generally, they follow the naming pattern <EmbJoprCompomentName>TestBase. These sub-classes contain test methods "stubs". They should provide at least general test abstraction, but usually they will also perform most of the test, as far as this code would be the same for AS 4 and AS 5.

The differences between tests for AS 4 and AS 5 are coded in subclasses of the *TestBase classes in the as4 and as5 packages. In case there's no difference between behavior of AS 4 and AS 5, the sub-class is not needed to override corresponding method; but, for clarity, it's better to override and explicitly call super.testSomething() method.

AS 4 and AS 5 test co-design procedure

• One of us, let's call him Lumberer, will put new test to as4 or as5 package (assuming that it builds and passes).
• Other of us, let's call him Carpenter, will review it, try it to run against the other AS version and manualy check the differences between EmbJopr for AS 4/5.
  • If there's no difference, the test method can be copied to the *TestBase class.
    • In the as#.*Test class, it can be removed, or replaced with trivial override:

      @Override protected void testSomething() { super.testSomething(); }

  • If there is a small difference, Carpenter will create an _abstract_ helper method, and move the part of the code which he needs to change to Lumberer's corresponding as# package. Then, carpenter will also override it in his/her as# package.
    • Lumberer can be aware of the difference in advance, so he can eventually create abtract helper method and override it in the Carpenter's package with a method just throwing UnsupportedOperationException.
    • Carpenter can also use simple if( isEap4() ){ ... } else { ... }, but the drawback would be that we will have tests for two AS versions. What if future AS versions will be different again? But for trivial cases like client.click( isEap4() ? "Continue" : "Save" ) I thing this is ok and saves us some complexity.
  • If there's a big difference that is probably not easily splittable, Carpenter will create an abstract testMethod(), leave whole test code in Lumberer's package, and create his/her own implementation.
  • The same applies to various helper methods serving more tests. For the methods in EmbjoprTestCase, currently there's no subclassing, but I suppose those methods will be generic enough to work with both AS versions. If not, we will solve it later.

Test coverage & plan

See JBoss Jira JBQA-1938.

Datasource

JVM Metrics

JMX Server

ConnectionFactory Service

Messaging Service

APPLICATIONS

The application strategy:

• will verify the navigation to Applications and Application Types
• verify the information for each application type is accurately displayed by the management console after initial deploy and after content modifications
• verify application deployment phases - deploy, undeploy, redeploy - including hot deployments managed by the console
  
• verify management of deployed applications using a variety of archive packaging
  

WAR

EJB

Enterprise Application (EAR)

See also:

• Embedded J