Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-6358

WARN HeuristicMixedException due to replication timeout

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Major Major
    • None
    • 10.0.0.Final
    • Clustering
    • None

      This is a separate issue originally logged as part of JBEAP-794 (#4 in this comment).

      Seen in one ejb-ejbservlet scenario and in all http-session scenarios with REPL cache and SYNC replication.

      This issue occured after perf18 (or any other node) rejoined a cluster (after graceful shutdown, jvmkill or undeploy - it doesn't matter), but it does not seem to affect client:

      [JBossINF] [0m[33m03:23:38,465 WARN  [io.undertow.request] (default task-12) javax.transaction.HeuristicMixedException: org.infinispan.commons.CacheException: javax.transaction.HeuristicMixedException
      [JBossINF] 	at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:74)
      [JBossINF] 	at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:76)
      [JBossINF] 	at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:768)
      [JBossINF] 	at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:563)
      [JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:331)
      [JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
      [JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
      [JBossINF] 	at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
      [JBossINF] 	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
      [JBossINF] 	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
      [JBossINF] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      [JBossINF] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      [JBossINF] 	at java.lang.Thread.run(Thread.java:745)
      [JBossINF] Caused by: javax.transaction.HeuristicMixedException
      [JBossINF] 	at org.infinispan.transaction.tm.DummyTransaction.finishResource(DummyTransaction.java:439)
      [JBossINF] 	at org.infinispan.transaction.tm.DummyTransaction.commitResources(DummyTransaction.java:448)
      [JBossINF] 	at org.infinispan.transaction.tm.DummyTransaction.runCommit(DummyTransaction.java:321)
      [JBossINF] 	at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:108)
      [JBossINF] 	at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:71)
      [JBossINF] 	... 12 more
      [JBossINF] Caused by: javax.transaction.xa.XAException
      [JBossINF] 	at org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:213)
      [JBossINF] 	at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:159)
      [JBossINF] 	at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:114)
      [JBossINF] 	at org.infinispan.transaction.tm.DummyTransaction.finishResource(DummyTransaction.java:401)
      [JBossINF] 	... 16 more
      [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf19
      [JBossINF] 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
      [JBossINF] 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$175(JGroupsTransport.java:612)
      [JBossINF] 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
      [JBossINF] 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
      [JBossINF] 	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
      [JBossINF] 	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
      [JBossINF] 	at org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
      [JBossINF] 	at org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
      [JBossINF] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      [JBossINF] 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
      [JBossINF] 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      [JBossINF] 	... 3 more
      

      The issue seems to have some relation to JBEAP-3779, JBEAP-3780, JBEAP-3781 as it (partially) happens in the same scenarios and numbers - see occurrences report

      Server link:
      http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-http-session-shutdown-repl-sync/18/console-perf18/

              pferraro@redhat.com Paul Ferraro
              mvinkler1@redhat.com Michal Vinkler
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: