Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-9909

Remoting channel has been closed errors after a node in cluster goes down (legacy ejb client)

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • 12.0.0.Beta1, 12.0.0.Final, 13.0.0.Final
    • EJB, Remoting
    • None

    Description

      We've been seeing this in our clustered ejb failover tests with the legacy (and only legacy) ejb client:

      2018/02/20 10:09:51:011 EST [ERROR][Runner - 950] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error getting response. <javax.ejb.EJBException: java.io.IOException: Channel Channel ID 88395849 (outbound) of Remoting connection 7f615772 to /10.16.90.54:8080 of endpoint "config-based-ejb-client-endpoint" <6949126f> has been closed>
              javax.ejb.EJBException: java.io.IOException: Channel Channel ID 88395849 (outbound) of Remoting connection 7f615772 to /10.16.90.54:8080 of endpoint "config-based-ejb-client-endpoint" <6949126f> has been closed
      	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:240)
      	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:185)
      	at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:148)
      	at com.sun.proxy.$Proxy2.getSerialAndIncrement(Unknown Source)
      	at org.jboss.smartfrog.clustering.ejb3.StatefulSBProcessorFactoryImplLegacyClient$EJB3RequestProcessor.processRequest(StatefulSBProcessorFactoryImplLegacyClient.java:79)
      	at org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:52)
      	at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:103)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.io.IOException: Channel Channel ID 88395849 (outbound) of Remoting connection 7f615772 to /10.16.90.54:8080 of endpoint "config-based-ejb-client-endpoint" <6949126f> has been closed
      	at org.jboss.ejb.client.remoting.ChannelAssociation$1.handleClose(ChannelAssociation.java:123)
      	at org.jboss.ejb.client.remoting.ChannelAssociation$1.handleClose(ChannelAssociation.java:115)
      	at org.jboss.remoting3.spi.SpiUtils.safeHandleClose(SpiUtils.java:50)
      	at org.jboss.remoting3.spi.AbstractHandleableCloseable$CloseHandlerTask.run(AbstractHandleableCloseable.java:520)
      	at org.jboss.remoting3.spi.AbstractHandleableCloseable.runCloseTask(AbstractHandleableCloseable.java:425)
      	at org.jboss.remoting3.spi.AbstractHandleableCloseable.closeComplete(AbstractHandleableCloseable.java:286)
      	at org.jboss.remoting3.remote.RemoteConnectionChannel.closeAction(RemoteConnectionChannel.java:508)
      	at org.jboss.remoting3.spi.AbstractHandleableCloseable.close(AbstractHandleableCloseable.java:150)
      	at org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver.disassociate(RemotingConnectionEJBReceiver.java:273)
      	at org.jboss.ejb.client.EJBClientContext.unregisterEJBReceiver(EJBClientContext.java:458)
      	at org.jboss.ejb.client.EJBClientContext.unregisterNodeEJBReceivers(EJBClientContext.java:497)
      	at org.jboss.ejb.client.remoting.EJBClientContextConnectionReconnectHandler.reconnect(EJBClientContextConnectionReconnectHandler.java:65)
      	at org.jboss.ejb.client.EJBClientContext$ReconnectAttempt.run(EJBClientContext.java:1474)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	... 1 more
      

      After having a look in the EAP 7.1 jobs, this was the case there as well, so it's not new. The pattern is similar, which is the reason it fell through our checkers, but there are no related NoSuchEjb exceptions here.

      These occur fairly often - that is hundreds of times for 2000 clients each making invocations every 4 seconds.

      Attachments

        Activity

          People

            Unassigned Unassigned
            rjanik@redhat.com Richard Janik
            Richard Janik Richard Janik
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: