Uploaded image for project: 'JBoss Cache'
  1. JBoss Cache
  2. JBCACHE-1592

org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn is very frequently in cluster

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Blocker
    • Resolution: Unresolved
    • Affects Version/s: 3.2.5.GA
    • Fix Version/s: None
    • Component/s: Locking
    • Labels:
      None

      Description

      When used in cluster,the following exception is throw frequently:

      org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn [/courseClass/360/male] after [5000] milliseconds for requestor [GlobalTransaction:<172.18.73.162:3430>:1843]! Lock held by [GlobalTransaction:<172.18.73.162:3428>:6416]
      at org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:159)
      at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:217)
      at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:186)
      at org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:101)
      at org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88)
      at org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
      at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
      at org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:150)
      at org.jboss.cache.interceptors.ReplicationInterceptor.visitPutKeyValueCommand(ReplicationInterceptor.java:107)
      at org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
      at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
      at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
      at org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65)
      at org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
      at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
      at org.jboss.cache.interceptors.TxInterceptor.replayModifications(TxInterceptor.java:501)
      at org.jboss.cache.interceptors.TxInterceptor.handleRemotePrepare(TxInterceptor.java:388)
      at org.jboss.cache.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:134)
      at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
      at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
      at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
      at org.jboss.cache.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:140)
      at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
      at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
      at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
      at org.jboss.cache.interceptors.InvocationContextInterceptor.visitPrepareCommand(InvocationContextInterceptor.java:106)
      at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
      at org.jboss.cache.interceptors.InterceptorChain.invokeRemote(InterceptorChain.java:316)
      at org.jboss.cache.commands.remote.ReplicateCommand.processSingleCommand(ReplicateCommand.java:139)
      at org.jboss.cache.commands.remote.ReplicateCommand.perform(ReplicateCommand.java:115)
      at org.jboss.cache.marshall.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:319)
      at org.jboss.cache.marshall.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:246)
      at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:637)
      at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:545)
      at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:368)
      at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:775)
      at org.jgroups.JChannel.up(JChannel.java:1336)
      at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:454)
      at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:486)
      at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:319)
      at org.jgroups.protocols.FRAG2.up(FRAG2.java:188)
      at org.jgroups.protocols.pbcast.GMS.up(GMS.java:820)
      at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
      at org.jgroups.protocols.UNICAST.up(UNICAST.java:328)
      at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:895)
      at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:708)
      at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
      at org.jgroups.protocols.FD.up(FD.java:284)
      at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:307)
      at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
      at org.jgroups.protocols.Discovery.up(Discovery.java:264)
      at org.jgroups.protocols.PING.up(PING.java:273)
      at org.jgroups.protocols.TP.passMessageUp(TP.java:1273)
      at org.jgroups.protocols.TP.access$100(TP.java:49)
      at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1826)
      at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1805)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:619)

      and following is my config:

      <jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:jboss:jbosscache-core:config:3.2">

      <locking isolationLevel="REPEATABLE_READ"
      lockParentForChildInsertRemove="false"
      lockAcquisitionTimeout="5000"
      nodeLockingScheme="mvcc"
      writeSkewCheck="true"
      useLockStriping="true"
      concurrencyLevel="10000"/>

      <clustering mode="replication" clusterName="JBossCache-cluster">
      <stateRetrieval timeout="5000" fetchInMemoryState="false" nonBlocking="false"/>
      <sync replTimeout="5000"/>
      <jgroupsConfig>
      <UDP discard_incompatible_packets="true" enable_bundling="false" enable_diagnostics="false" ip_ttl="2"
      loopback="false" max_bundle_size="64000" max_bundle_timeout="30" mcast_addr="224.0.0.1"
      mcast_port="45588" mcast_recv_buf_size="25000000" mcast_send_buf_size="640000"
      oob_thread_pool.enabled="true" oob_thread_pool.keep_alive_time="10000" oob_thread_pool.max_threads="4"
      oob_thread_pool.min_threads="1" oob_thread_pool.queue_enabled="true"
      oob_thread_pool.queue_max_size="10"
      oob_thread_pool.rejection_policy="Run" thread_naming_pattern="pl" thread_pool.enabled="true"
      thread_pool.keep_alive_time="30000" thread_pool.max_threads="25" thread_pool.min_threads="1"
      thread_pool.queue_enabled="true" thread_pool.queue_max_size="10" thread_pool.rejection_policy="Run"
      tos="8" ucast_recv_buf_size="20000000" ucast_send_buf_size="640000" use_concurrent_stack="true"
      use_incoming_packet_handler="true"/>
      <PING num_initial_members="3" timeout="2000"/>
      <MERGE2 max_interval="30000" min_interval="10000"/>
      <FD_SOCK/>
      <FD max_tries="5" shun="true" timeout="10000"/>
      <VERIFY_SUSPECT timeout="1500"/>
      <pbcast.NAKACK discard_delivered_msgs="true" gc_lag="0" retransmit_timeout="300,600,1200,2400,4800"
      use_mcast_xmit="false"/>
      <UNICAST timeout="300,600,1200,2400,3600"/>
      <pbcast.STABLE desired_avg_gossip="50000" max_bytes="400000" stability_delay="1000"/>
      <pbcast.GMS join_timeout="5000" print_local_addr="true" shun="false" view_ack_collection_timeout="5000"
      view_bundling="true"/>
      <FRAG2 frag_size="60000"/>
      <pbcast.STREAMING_STATE_TRANSFER/>
      <pbcast.FLUSH timeout="0"/>
      </jgroupsConfig>
      </clustering>

      </jbosscache>

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                manik Manik Surtani
                Reporter:
                likebean l jw
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated: