Uploaded image for project: 'Application Server 3  4  5 and 6'
  1. Application Server 3 4 5 and 6
  2. JBAS-5943

JRMPInvokerProxyHA should not failover if PooledInvokerProxy.invoke() throws java.rmi.ConnectException with cause of java.io.EOFException

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Minor Minor
    • JBossAS-5.0.0.GA
    • JBossAS-4.0.5.GA, JBossAS-5.0.0.CR1, JBossAS-4.2.3.GA
    • Clustering
    • None
    • Low
    • Workaround Exists
    • Hide

      Stop the master/singleton node to correct the 02 nodes familyClusterInfo (list of valid servers in the cluster). You can restart the 01 node after the cluster membership has been updated.

      Show
      Stop the master/singleton node to correct the 02 nodes familyClusterInfo (list of valid servers in the cluster). You can restart the 01 node after the cluster membership has been updated.

      In a five node cluster

      {01, 02, 03, 04, 05}

      , the 01 node was running as the master (all singleton services are running here). The 02 node was making a stateless session bean invocation to the 01 (singleton) node via the PooledInvokerHA transport. Some type of transient failure occurred on the pooled socket connection. The java.io.EOFException occurred while waiting to read the response from the singleton node. The PooledInvokerProxy.invoke wraps all Exceptions as java.rmi.ConnectException which is caught by JRMPInvokerProxyHA. JRMPInvokerProxyHA currently will treat all java.rmi.ConnectException as a reason to fail-over to a different node, however, that is wrong for the case of EOFException (only the socket connection is broken, not the 01 node). As part of the failover handling, the 01 node is removed from the 02 nodes familyClusterInfo which means that the singleton node will no longer be accessible to this singleton session bean on the 02 node.

      I was able to recreate this with an application test running on my laptop (with two instances of as running) but haven't reproduced the failure with a unit test yet.

      Exception stacktrace:
      java.rmi.ConnectException: Failed to communicate; nested exception is:
      java.io.EOFException
      at org.jboss.invocation.pooled.interfaces.PooledInvokerProxy.invoke(PooledInvokerProxy.java:628)
      at org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA.invoke(JRMPInvokerProxyHA.java:200)
      at org.jboss.invocation.InvokerInterceptor.invokeInvoker(InvokerInterceptor.java:331)
      at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:194)
      at org.jboss.proxy.ejb.RetryInterceptor.invoke(RetryInterceptor.java:176)
      ... [more]
      Caused by: java.io.EOFException
      at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2498)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1273)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348)
      at org.jboss.invocation.pooled.interfaces.PooledInvokerProxy.invoke(PooledInvokerProxy.java:611)
      ... 102 more

              smarlow1@redhat.com Scott Marlow
              smarlow1@redhat.com Scott Marlow
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

                Created:
                Updated:
                Resolved: