Failures/regressions with WSTF Scenario 3 (JAXWS + WSA) intreop tests on AS trunk/CXF and AS 6.0.0.M3/Native

The WSTF interop test scenarios include two scenarios for checking WS interopability, SC002 and Sc003:

http://www.wstf.org/docs/scenarios/sc002/sc002.xml
http://www.wstf.org/docs/scenarios/sc003/sc003.xml

The first tests basic JaxWS functionality. The second tests JaxWS with WSA functionality.

We have two implementations which both previously worked on AS 4 and AS 5.0.1 using JBossWS Native. For historical reasons the code for both tests is located in the TS repo:

http://anonsvn.jboss.org/repos/labs/labs/jbosstm/workspace/interop/WSTFSC02-interop/

I updated these in response to JBWS-3069 to try so as to make it possible to run them on

i) AS 6.0.0.M3 using WS-Native

ii) the current AS trunk using WS-CXF and including CXF 2.2.9 patched the fixes supplied for JBWS-3069

There are regressions for four of the eight Sc003 Asynchronous test cases on 6.0.0.M3/Native i.e. tests 3, 4, 7 and 8

There are failures for seven of these eight test cases on trunk/CXF i.e. test 2, ... 7

The eight Async tests in our implementation check what happens when a JaxWS request is provided with a non-null/anonymous FaultTo and ReplyTo address in various combinations of cases. These are now a subset of the cases defined in the test spec. In particular we always supply both ReplyTo and FaultTo (they always identify the same response/fault handling service). The combinations of cases we implement are:

1 MEP= OneWay, Exception=No, MU=no
2 MEP= OneWay, Exception=Yes, MU=no
3 MEP= RPC, Exception=No, MU=no
4 MEP= RPC, Exception=Yes, MU=no
5 MEP= OneWay, Exception=No, MU=yes
6 MEP= OneWay, Exception=Yes, MU=yes
7 MEP= RPC, Exception=No, MU=yes
8 MEP= RPC, Exception=Yes, MU=yes

Thes cases are explained as follows:

MEP:
The service implements a one way message which merely saves an input value for later use. It also implements an RPC style message which accepts an input value and returns a response. This field indicates which request is made.

Exception:
Depending on the value of an input parameter to the service request the service implementation bean will either complete correctly or throw a WebServiceException. This field determines which behaviour is employed in the test.

MU:
The request may include a MustUnderstand header which will not be understood. The service is not expected to understand this header so the WS stack should return a MustUnderstand fault in cases where MU is yes and it is appropriate to return a fault.

The tests are now slightly out sync with the latest test spec which no longer defines Fault dispatch for OneWay messages when ReplyTo or FaultTo is specified. This is a shame as XTS sends OneWay requests which use a MU header for the transaction context and needs to know when the other end cannot understand this header. Anyway, these tests now have different outcomes and it needs to be decided whether this is important to us or not.

The outcomes expected by the tests (and met on 5.0.1) are

1) No fault or reply returned to response endpoint
2) "FaultingNotify" Fault dispatched to response endpoint
3) Result dispatched to response endpoint, no result or fault on back channel
4) "Server" Fault with "FaultingNotify" fault string dispatched to response endpoint, no result or fault on back channel
5) "MustUnderstand" fault dispatched to response endpoint
6) "MustUnderstand" fault dispatched to response endpoint, no result or fault on back channel 87) "MustUnderstand" fault dispatched to response endpoint, no result or fault on back channel
7) "MustUnderstand" fault dispatched to response endpoint, no result or fault on back channel

The actual results for trunk/CXF are
1) as expected
2) No fault dispatched
3) Result dispatched on back channel
4) "Server" fault dispatched to response endpoint but with incorrect Action
5) No fault dispatched to response endpoint
6) No fault dispatched to response endpoint
7) "MustUnderstand" fault dispatched to response endpoint but with incorrect Action
8) "MustUnderstand" fault dispatched to response endpoint but with incorrect Action

The actual results for 6.0.0.M3/Native are
1) as expected
2) as expected
3) Result dispatched on back channel
4) Client side unmarshalling exception " Unsupported content type: text/html"
5) as expected
6) as expected
7) Client side unmarshalling exception " Unsupported content type: text/html"
8) Client side unmarshalling exception " Unsupported content type: text/html"

It may be that the tests need revising or modifying to cover more/different scenarios in the test spec so we can do a more full check of JaxWS interoperability. Whatever is done with them they really need to be kept up to date with the WS releases. The client needs to be run regularly against its own service implementation on the trunk CXF and Native releases to make sure our releases are at least internally consistent with expectations. . This could easily be automated via a hudson job as has been done for the XTS unit tests.

I think the job of maintaining, running and monitoring these tests internally needs to be taken over by the WS team. I am still very happy to install and check the Sc002/3 web app on the public JBossTS server used for interop testing where we expose the Sc007 XTS interop test service and client.