Hi Howard,

I tried following test scenario:

1. Start cluster of 4 nodes - A,B,C,D
2. Start 4 publishers and 4 subscribers on one clustered topic - clients use ClusteredConnectionFactory with transaction acknowledge session.
3. Kill server A
4. Kill server B
5. Kill server C
6. Start A
7. Kill D
8. Stop producers

Clients should be reconnected to other nodes in cluster during killing servers but sometimes happens that client won't get reconnected to other node and ends with exception. I see following exception in client log:

<b>Consumer for node: 10.16.90.66:1099 and topic /topic/testDistributedTopic ENDED because of exception. Number of consumed messages is 3700 , receiveTimeout=120000, contextProperties=

{java.naming.provider.url=jnp://10.16.90.66:1099, java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory, java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces}

</b>
Print the exception:
Failed to invoke
org.jboss.jms.exception.MessagingTransactionRolledBackException: Failed to invoke
at org.jboss.jms.tx.ResourceManager.commitLocal(ResourceManager.java:225)
at org.jboss.jms.client.container.SessionAspect.handleCommit(SessionAspect.java:582)
at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect_z_handleCommit_899770473.invoke(SessionAspect_z_handleCommit_899770473.java)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:172)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.jms.client.delegate.ClientSessionDelegate.commit(ClientSessionDelegate.java)
at org.jboss.jms.client.JBossSession.commit(JBossSession.java:164)
at org.jboss.smartfrog.jbm.SimpleReceiverTopicTransacted.clientLogic(SimpleReceiverTopicTransacted.java:165)
at org.jboss.smartfrog.jbm.AJMSClient.run(AJMSClient.java:287)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.jboss.jms.exception.MessagingJMSException: Failed to invoke
at org.jboss.jms.client.delegate.DelegateSupport.handleThrowable(DelegateSupport.java:271)
at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:205)
at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)
at org.jboss.jms.client.delegate.ClientConnectionDelegate.org$jboss$jms$client$delegate$ClientConnectionDelegate$sendTransaction$aop(ClientConnectionDelegate.java:239)
at org.jboss.jms.client.delegate.ClientConnectionDelegate$sendTransaction_N3268650789275322226.invokeTarget(ClientConnectionDelegate$sendTransaction_N3268650789275322226.java)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:114)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:172)
at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.jms.client.delegate.ClientConnectionDelegate.sendTransaction(ClientConnectionDelegate.java)
at org.jboss.jms.tx.ResourceManager.commitLocal(ResourceManager.java:206)
... 11 more
Caused by: java.lang.Exception: Can not make remoting client invocation due to not being connected to server.
at org.jboss.remoting.Client.invoke(Client.java:2075)
at org.jboss.remoting.Client.invoke(Client.java:879)
at org.jboss.remoting.Client.invoke(Client.java:867)
at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)
... 23 more

Please see hudson job:
https://hudson.qa.jboss.com/hudson/view/EAP5-JBM-Failover/job/eap-5-topic-trans-kill/4/

Could you look at it, please? Give me know if you need more information.

Thank you,

Mirek