Ours is a J2EE application (clustered across 4 nodes) internally using Infinispan Cache (Ver 6.0.2) with JGroups (Ver 3.4.1) as transport layer of our cache.

Application had been running perfectly fine, but suddenly on one node, one cache-put got stuck with below stack trace. (Cause being blocked on org.jgroups.protocols.FlowControl$Credit.decrementIfEnoughCredits(FlowControl.java:567)).

Now all further puts are getting stuck - blocked - on the same line on the same node. Other nodes are working fine in this respect.

Currently, while raising the issue, the threads are still in stuck state. And the first stuck thread is stuck from last 27 hours.

Stack Trace:
"[STUCK] ExecuteThread: '37' for queue: 'weblogic.kernel.Default (self-tuning)'" #107862 daemon prio=1 os_prio=0 tid=0x00007f5964373000 nid=0x75a9 in Object.wait() [0x00007f57d596b000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.jgroups.protocols.FlowControl$Credit.decrementIfEnoughCredits(FlowControl.java:567)

• locked <0x00007f5e64893168> (a org.jgroups.protocols.FlowControl$Credit)
  at org.jgroups.protocols.UFC.handleDownMessage(UFC.java:121)
  at org.jgroups.protocols.FlowControl.down(FlowControl.java:330)
  at org.jgroups.protocols.FlowControl.down(FlowControl.java:340)
  at org.jgroups.protocols.FRAG2.down(FRAG2.java:136)
  at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:1024)
  at org.jgroups.JChannel.down(JChannel.java:760)
  at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:683)
  at org.jgroups.blocks.RequestCorrelator.sendUnicastRequest(RequestCorrelator.java:202)
  at org.jgroups.blocks.UnicastRequest.sendRequest(UnicastRequest.java:43)
  at org.jgroups.blocks.Request.execute(Request.java:83)
  at org.jgroups.blocks.MessageDispatcher.sendMessage(MessageDispatcher.java:399)
  at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:353)
  at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:167)
  at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:521)
  at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:281)
  at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.handleNonTxWriteCommand(BaseDistributionInterceptor.java:233)
  at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.visitPutKeyValueCommand(NonTxDistributionInterceptor.java:72)
  at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70)
  at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
  at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:321)
  at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:402)
  at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:164)
  at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70)
  at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
  at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:68)
  at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70)
  at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
  at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
  at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:32)
  at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70)
  at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
  at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
  at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:32)
  at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70)
  at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
  at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:219)
  at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:141)
  at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70)
  at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
  at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148)
  at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:134)
  at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70)
  at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
  at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)
  at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)
  at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:32)
  at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70)
  at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
  at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1306)
  at org.infinispan.CacheImpl.putInternal(CacheImpl.java:878)
  at org.infinispan.CacheImpl.put(CacheImpl.java:870)
  at org.infinispan.DecoratedCache.put(DecoratedCache.java:401)
  at com.nucleus.finnone.pro.cache.common.NeutrinoCache.put(NeutrinoCache.java:18)
  at com.nucleus.config.persisted.service.ConfigurationServiceImpl.getConfigurationGroupFor(ConfigurationServiceImpl.java:478)

Note: Seems related to below-
https://issues.jboss.org/browse/JGRP-1675
https://issues.jboss.org/browse/ISPN-3645