Uploaded image for project: 'JGroups'
  1. JGroups
  2. JGRP-1314

FD_SOCK port conflicts with concurrently connected channels

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 2.12.2, 3.0
    • 2.12.1
    • None
    • Hide

      Synchronize channel connection

      Show
      Synchronize channel connection

      Because of the high level of concurrency in AS7, I'm seeing occasional socket exceptions in FD_SOCK when trying to start multiple channels using a shared transport. It appears that there are some thread safety issues with the port increment logic within FD_SOCK.
      Until JGRP-790 is complete, measures need to be taken to ensure that multiple concurrently connected channels do not select the same port on which to open a server socket.

      Here's a sample stack trace:

      10:01:26,716 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-7) MSC00001: Failed to start service jboss.infinispan.cluster: org.jboss.msc.service.StartException in service jboss.infinispan.cluster: Failed to start service
      at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1696)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [:1.6.0_22]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [:1.6.0_22]
      at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
      Caused by: org.infinispan.CacheException: Unable to invoke method public abstract void org.infinispan.remoting.transport.Transport.start() on object
      at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:174)
      at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:889)
      at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:687)
      at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:589)
      at org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:137)
      at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:137)
      at org.infinispan.CacheDelegate.start(CacheDelegate.java:323)
      at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:516)
      at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:439)
      at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:77)
      at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:66)
      at org.jboss.as.clustering.infinispan.subsystem.EmbeddedCacheManagerService.start(EmbeddedCacheManagerService.java:171)
      at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1675)
      ... 3 more
      Caused by: java.lang.reflect.InvocationTargetException
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [:1.6.0_22]
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [:1.6.0_22]
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_22]
      at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_22]
      at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:171)
      ... 15 more
      Caused by: java.lang.IllegalArgumentException: failed to start server socket
      at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:295)
      at org.jgroups.protocols.FD.down(FD.java:310)
      at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:70)
      at org.jgroups.protocols.BARRIER.down(BARRIER.java:90)
      at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:634)
      at org.jgroups.protocols.UNICAST.down(UNICAST.java:443)
      at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:330)
      at org.jgroups.protocols.VIEW_SYNC.down(VIEW_SYNC.java:186)
      at org.jgroups.protocols.pbcast.GMS.down(GMS.java:926)
      at org.jgroups.protocols.FlowControl.down(FlowControl.java:351)
      at org.jgroups.protocols.FlowControl.down(FlowControl.java:351)
      at org.jgroups.protocols.FRAG2.down(FRAG2.java:155)
      at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.down(STREAMING_STATE_TRANSFER.java:324)
      at org.jgroups.protocols.pbcast.FLUSH.handleConnect(FLUSH.java:302)
      at org.jgroups.protocols.pbcast.FLUSH.down(FLUSH.java:263)
      at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:1081)
      at org.jgroups.JChannel.downcall(JChannel.java:1680)
      at org.jgroups.JChannel.connect(JChannel.java:425)
      at org.jgroups.JChannel.connect(JChannel.java:388)
      at org.infinispan.remoting.transport.jgroups.JGroupsTransport.startJGroupsChannelIfNeeded(JGroupsTransport.java:171)
      at org.infinispan.remoting.transport.jgroups.JGroupsTransport.start(JGroupsTransport.java:155)
      ... 20 more

      Caused by: java.net.SocketException: Address already in use
      at java.net.PlainSocketImpl.socketListen(Native Method) [:1.6.0_22]
      at java.net.AbstractPlainSocketImpl.listen(AbstractPlainSocketImpl.java:365) [:1.6.0_22]
      at java.net.ServerSocket.bind(ServerSocket.java:337) [:1.6.0_22]
      at java.net.ServerSocket.<init>(ServerSocket.java:202) [:1.6.0_22]
      at org.jgroups.util.DefaultSocketFactory.createServerSocket(DefaultSocketFactory.java:49)
      at org.jgroups.util.Util.createServerSocket(Util.java:3046)
      at org.jgroups.protocols.FD_SOCK.startServerSocket(FD_SOCK.java:561)
      at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:292)
      ... 40 more

              rhn-engineering-bban Bela Ban
              pferraro@redhat.com Paul Ferraro
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: