Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-9909

Remoting channel has been closed errors after a node in cluster goes down (legacy ejb client)

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 12.0.0.Beta1, 12.0.0.Final, 13.0.0.Final
    • EJB, Remoting
    • None

      We've been seeing this in our clustered ejb failover tests with the legacy (and only legacy) ejb client:

      2018/02/20 10:09:51:011 EST [ERROR][Runner - 950] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error getting response. <javax.ejb.EJBException: java.io.IOException: Channel Channel ID 88395849 (outbound) of Remoting connection 7f615772 to /10.16.90.54:8080 of endpoint "config-based-ejb-client-endpoint" <6949126f> has been closed>
              javax.ejb.EJBException: java.io.IOException: Channel Channel ID 88395849 (outbound) of Remoting connection 7f615772 to /10.16.90.54:8080 of endpoint "config-based-ejb-client-endpoint" <6949126f> has been closed
      	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:240)
      	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:185)
      	at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:148)
      	at com.sun.proxy.$Proxy2.getSerialAndIncrement(Unknown Source)
      	at org.jboss.smartfrog.clustering.ejb3.StatefulSBProcessorFactoryImplLegacyClient$EJB3RequestProcessor.processRequest(StatefulSBProcessorFactoryImplLegacyClient.java:79)
      	at org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:52)
      	at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:103)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.io.IOException: Channel Channel ID 88395849 (outbound) of Remoting connection 7f615772 to /10.16.90.54:8080 of endpoint "config-based-ejb-client-endpoint" <6949126f> has been closed
      	at org.jboss.ejb.client.remoting.ChannelAssociation$1.handleClose(ChannelAssociation.java:123)
      	at org.jboss.ejb.client.remoting.ChannelAssociation$1.handleClose(ChannelAssociation.java:115)
      	at org.jboss.remoting3.spi.SpiUtils.safeHandleClose(SpiUtils.java:50)
      	at org.jboss.remoting3.spi.AbstractHandleableCloseable$CloseHandlerTask.run(AbstractHandleableCloseable.java:520)
      	at org.jboss.remoting3.spi.AbstractHandleableCloseable.runCloseTask(AbstractHandleableCloseable.java:425)
      	at org.jboss.remoting3.spi.AbstractHandleableCloseable.closeComplete(AbstractHandleableCloseable.java:286)
      	at org.jboss.remoting3.remote.RemoteConnectionChannel.closeAction(RemoteConnectionChannel.java:508)
      	at org.jboss.remoting3.spi.AbstractHandleableCloseable.close(AbstractHandleableCloseable.java:150)
      	at org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver.disassociate(RemotingConnectionEJBReceiver.java:273)
      	at org.jboss.ejb.client.EJBClientContext.unregisterEJBReceiver(EJBClientContext.java:458)
      	at org.jboss.ejb.client.EJBClientContext.unregisterNodeEJBReceivers(EJBClientContext.java:497)
      	at org.jboss.ejb.client.remoting.EJBClientContextConnectionReconnectHandler.reconnect(EJBClientContextConnectionReconnectHandler.java:65)
      	at org.jboss.ejb.client.EJBClientContext$ReconnectAttempt.run(EJBClientContext.java:1474)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	... 1 more
      

      After having a look in the EAP 7.1 jobs, this was the case there as well, so it's not new. The pattern is similar, which is the reason it fell through our checkers, but there are no related NoSuchEjb exceptions here.

      These occur fairly often - that is hundreds of times for 2000 clients each making invocations every 4 seconds.

              Unassigned Unassigned
              rjanik@redhat.com Richard Janik
              Richard Janik Richard Janik
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: