Uploaded image for project: 'JBoss Naming'
  1. JBoss Naming
  2. JBNAME-59

HA-JNDI java.rmi.UnmarshallException doesn't failover

XMLWordPrintable

    • Hide

      Use the attached byteman script.

      JAVA_OPTS=-javaagent:$BYTEMAN_HOME/lib/byteman.jar=script:/path/to/pauseHARMIServerImplinvoke.btm

      (This script just pauses at the right spot to make it easier to trigger the bug.
      Without the script, just restart one member of the cluster repeatedly while hitting HA-JNDI until the
      timing is just right to trigger it).

      From a standalone client configured with jndi.properties to hit HA-JNDI, run:

      InitialContext context = new InitialContext()
      context.lookup ( ... );

      Stop the server while it's paused.

      Show
      Use the attached byteman script. JAVA_OPTS=-javaagent:$BYTEMAN_HOME/lib/byteman.jar=script:/path/to/pauseHARMIServerImplinvoke.btm (This script just pauses at the right spot to make it easier to trigger the bug. Without the script, just restart one member of the cluster repeatedly while hitting HA-JNDI until the timing is just right to trigger it). From a standalone client configured with jndi.properties to hit HA-JNDI, run: InitialContext context = new InitialContext() context.lookup ( ... ); Stop the server while it's paused.
    • Workaround Exists
    • Hide

      Catch the exception and retry from client code.

      Show
      Catch the exception and retry from client code.

      During intensive HA-JNDI failover testing (constantly running "new InitialContext().lookup(...)"
      while starting and stopping JBoss instances, sometimes an HA-JNDI lookup just before the server goes down is
      throwing an exception without trying to fail over to the other HA-JNDI instance:

      ...
      Caused by: javax.naming.CommunicationException [Root exception is java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is:
      java.net.SocketException: Connection reset]
      at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:863)
      at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:710)
      at javax.naming.InitialContext.lookup(InitialContext.java:351)
      at xxx.CustomCode (CustomCode.java:XXX)
      ... 25 more
      Caused by: java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is:
      java.net.SocketException: Connection reset
      at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:203)
      at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:126)
      at org.jboss.ha.framework.server.HARMIServerImpl_Stub.invoke(Unknown Source)
      at org.jboss.ha.framework.interfaces.HARMIClient.invokeRemote(HARMIClient.java:225)
      at org.jboss.ha.framework.interfaces.HARMIClient.invoke(HARMIClient.java:327)
      at $Proxy4.lookup(Unknown Source)
      at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:750)
      ... 28 more
      Caused by: java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:168)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
      at java.io.DataInputStream.readByte(DataInputStream.java:241)
      at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:189)
      ... 34 more

      Also a similar exception:
      Caused by: java.net.SocketException: socket write error: Connection reset by peer.
      at jrockit.net.SocketNativeIO.writeBytesPinned(Native Method)
      at jrockit.net.SocketNativeIO.socketWrite(SocketNativeIO.java:73)
      at java.net.SocketOutputStream.socketWrite0(SocketOutputStream.java)
      at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
      at java.net.SocketOutputStream.write(SocketOutputStream.java:137)
      at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
      at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
      at java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputStream.java:1632)
      at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:667)
      at sun.rmi.transport.StreamRemoteCall.releaseOutputStream(StreamRemoteCall.java:92)
      at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:188)
      at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:126)
      at org.jboss.ha.framework.server.HARMIServerImpl_Stub.invoke(Unknown Source)
      at org.jboss.ha.framework.interfaces.HARMIClient.invokeRemote(HARMIClient.java:225)
      at org.jboss.ha.framework.interfaces.HARMIClient.invoke(HARMIClient.java:327)
      at $Proxy4.lookup(Unknown Source)
      at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:750)
      at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:710)
      at javax.naming.InitialContext.lookup(InitialContext.java:351)
      ...

      Also
      javax.naming.CommunicationException [Root exception is java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is:
      java.io.EOFException]
      ...

      Root cause is that java.rmi.UnmarshallException is not caught as an exception eligible for failover.

        1. pauseHARMIServerImplinvoke.btm
          0.2 kB
          Dennis Reed

              rhn-support-dereed Dennis Reed
              rhn-support-dereed Dennis Reed
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

                Created:
                Updated:
                Resolved: