Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-3782

WARN HeuristicMixedException due to replication timeout

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 7.1.0.DR13
    • 7.0.0.ER6, 7.0.1.CR2, 7.1.0.DR9
    • Clustering
    • None

    Description

      This is a separate issue originally logged as part of JBEAP-794 (#4 in this comment).

      Seen in one ejb-ejbservlet scenario and in all http-session scenarios with REPL cache and SYNC replication.

      This issue occured after perf18 (or any other node) rejoined a cluster (after graceful shutdown, jvmkill or undeploy - it doesn't matter), but it does not seem to affect client:

      [JBossINF] [0m[33m03:23:38,465 WARN  [io.undertow.request] (default task-12) javax.transaction.HeuristicMixedException: org.infinispan.commons.CacheException: javax.transaction.HeuristicMixedException
[JBossINF] 	at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:74)
[JBossINF] 	at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:76)
[JBossINF] 	at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:768)
[JBossINF] 	at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:563)
[JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:331)
[JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
[JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
[JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
[JBossINF] 	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
[JBossINF] 	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
[JBossINF] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] 	at java.lang.Thread.run(Thread.java:745)
[JBossINF] Caused by: javax.transaction.HeuristicMixedException
[JBossINF] 	at org.infinispan.transaction.tm.DummyTransaction.finishResource(DummyTransaction.java:439)
[JBossINF] 	at org.infinispan.transaction.tm.DummyTransaction.commitResources(DummyTransaction.java:448)
[JBossINF] 	at org.infinispan.transaction.tm.DummyTransaction.runCommit(DummyTransaction.java:321)
[JBossINF] 	at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:108)
[JBossINF] 	at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:71)
[JBossINF] 	... 12 more
[JBossINF] Caused by: javax.transaction.xa.XAException
[JBossINF] 	at org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
[JBossINF] 	at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
[JBossINF] 	at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:114)
[JBossINF] 	at org.infinispan.transaction.tm.DummyTransaction.finishResource(DummyTransaction.java:401)
[JBossINF] 	... 16 more
[JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf19
[JBossINF] 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
[JBossINF] 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$175(JGroupsTransport.java:612)
[JBossINF] 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
[JBossINF] 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
[JBossINF] 	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
[JBossINF] 	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
[JBossINF] 	at org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
[JBossINF] 	at org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
[JBossINF] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[JBossINF] 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[JBossINF] 	... 3 more

      The issue seems to have some relation to JBEAP-3779, JBEAP-3780, JBEAP-3781 as it (partially) happens in the same scenarios and numbers - see occurrences report

      Server link:
      http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-http-session-shutdown-repl-sync/18/console-perf18/

      Attachments

        Issue Links

          is caused by

          Bug - A problem that impairs or prevents the functions of the product. ISPN-7064 RPC to leaver times out instead of finishing immediately

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed
          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. WFLY-6358 WARN HeuristicMixedException due to replication timeout

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is incorporated by

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. JBEAP-7048 Upgrade Infinispan to 8.2.5.Final

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Verified

          Activity

            People

              pferraro@redhat.com Paul Ferraro
              mvinkler1@redhat.com Michal Vinkler
              Daniel Cihak Daniel Cihak
              Daniel Cihak Daniel Cihak
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: