Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-9909

Remoting channel has been closed errors after a node in cluster goes down (legacy ejb client)

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 12.0.0.Beta1, 12.0.0.Final, 13.0.0.Final
    • Fix Version/s: None
    • Component/s: EJB, Remoting
    • Labels:
      None

      Description

      We've been seeing this in our clustered ejb failover tests with the legacy (and only legacy) ejb client:

      2018/02/20 10:09:51:011 EST [ERROR][Runner - 950] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error getting response. <javax.ejb.EJBException: java.io.IOException: Channel Channel ID 88395849 (outbound) of Remoting connection 7f615772 to /10.16.90.54:8080 of endpoint "config-based-ejb-client-endpoint" <6949126f> has been closed>
              javax.ejb.EJBException: java.io.IOException: Channel Channel ID 88395849 (outbound) of Remoting connection 7f615772 to /10.16.90.54:8080 of endpoint "config-based-ejb-client-endpoint" <6949126f> has been closed
      	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:240)
      	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:185)
      	at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:148)
      	at com.sun.proxy.$Proxy2.getSerialAndIncrement(Unknown Source)
      	at org.jboss.smartfrog.clustering.ejb3.StatefulSBProcessorFactoryImplLegacyClient$EJB3RequestProcessor.processRequest(StatefulSBProcessorFactoryImplLegacyClient.java:79)
      	at org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:52)
      	at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:103)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.io.IOException: Channel Channel ID 88395849 (outbound) of Remoting connection 7f615772 to /10.16.90.54:8080 of endpoint "config-based-ejb-client-endpoint" <6949126f> has been closed
      	at org.jboss.ejb.client.remoting.ChannelAssociation$1.handleClose(ChannelAssociation.java:123)
      	at org.jboss.ejb.client.remoting.ChannelAssociation$1.handleClose(ChannelAssociation.java:115)
      	at org.jboss.remoting3.spi.SpiUtils.safeHandleClose(SpiUtils.java:50)
      	at org.jboss.remoting3.spi.AbstractHandleableCloseable$CloseHandlerTask.run(AbstractHandleableCloseable.java:520)
      	at org.jboss.remoting3.spi.AbstractHandleableCloseable.runCloseTask(AbstractHandleableCloseable.java:425)
      	at org.jboss.remoting3.spi.AbstractHandleableCloseable.closeComplete(AbstractHandleableCloseable.java:286)
      	at org.jboss.remoting3.remote.RemoteConnectionChannel.closeAction(RemoteConnectionChannel.java:508)
      	at org.jboss.remoting3.spi.AbstractHandleableCloseable.close(AbstractHandleableCloseable.java:150)
      	at org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver.disassociate(RemotingConnectionEJBReceiver.java:273)
      	at org.jboss.ejb.client.EJBClientContext.unregisterEJBReceiver(EJBClientContext.java:458)
      	at org.jboss.ejb.client.EJBClientContext.unregisterNodeEJBReceivers(EJBClientContext.java:497)
      	at org.jboss.ejb.client.remoting.EJBClientContextConnectionReconnectHandler.reconnect(EJBClientContextConnectionReconnectHandler.java:65)
      	at org.jboss.ejb.client.EJBClientContext$ReconnectAttempt.run(EJBClientContext.java:1474)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	... 1 more
      

      After having a look in the EAP 7.1 jobs, this was the case there as well, so it's not new. The pattern is similar, which is the reason it fell through our checkers, but there are no related NoSuchEjb exceptions here.

      These occur fairly often - that is hundreds of times for 2000 clients each making invocations every 4 seconds.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                rjanik Richard Janik
                Tester:
                Richard Janik
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: