Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-21044

Not recovered transactions after crash of server with MDB

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Blocker Blocker
    • 38.0.1.Final
    • 38.0.0.Beta1
    • JMS
    • None
    • Hide

      Steps to reproduce:

      git clone git@gitlab.cee.redhat.com:mnovak/messaging-testsuite.git
      cd messaging-testsuite/scripts/
      git checkout missing-messages
      
      groovy -DEAP_ZIP_URL=<path-to-wildfly-server-zip> PrepareServers7.groovy
      
      export WORKSPACE=$PWD
      export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap
      export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap
      export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap
      export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap
      
      cd ../jboss-hornetq-testsuite/
      mvn clean install -Dtest=Lodh2TestCase#testSimpleLodh2killMdbLocalRemote -Deap7.org.jboss.qa.hornetq.apps.clients.version=8.1760015969-8.x-8524-202510090117-SNAPSHOT -Dversion.artemis=2.42.0 -Dsurefire.failIfNoSpecifiedTests=false
      

       

       

      Show
      Steps to reproduce: git clone git@gitlab.cee.redhat.com:mnovak/messaging-testsuite.git cd messaging-testsuite/scripts/ git checkout missing-messages groovy -DEAP_ZIP_URL=<path-to-wildfly-server-zip> PrepareServers7.groovy export WORKSPACE=$PWD export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap cd ../jboss-hornetq-testsuite/ mvn clean install -Dtest=Lodh2TestCase#testSimpleLodh2killMdbLocalRemote -Deap7.org.jboss.qa.hornetq.apps.clients.version=8.1760015969-8.x-8524-202510090117-SNAPSHOT -Dversion.artemis=2.42.0 -Dsurefire.failIfNoSpecifiedTests= false    
    • ---
    • ---

      There can be unrecovered transactions on Artemis when remote server with MDB crashed and is restarted again. 

      Test Scenario:

      • Start Cluster A (Node 1 and Node 3) and deploy queue OutQueue.
      • Start Cluster B (Node 2 and Node 4) and deploy queue InQueue.
      • Start a producer that sends 1000 mixed-size (large and normal) messages to InQueue.
      • Wait for the producer to complete message sending.
      • Deploy MDBs on Node 2 and Node 4 which consume messages from InQueue (node 2 and 4) and send to remote OutQueue (node 1 and 3) in XA transaction.
      • Kill Node 2 while MDBs are actively processing messages.
      • Restart Node 2.
      • Wait until all messages have been processed. Provide enough time for recovery of XA transactions.
      • Start a consumer to consume all messages from OutQueue on node 1.

      Expected result: All messages will be consumed from OutQueue from node 1.
      Actual results: Some messages are missing. There are unrecovered Artemis transactions on nodes 1 and 3.

      Investigation showed that there are some (Artemis) transactions in prepared state on node 1 or 3 (holding those messages). There are also warnings during periodic recovery on node 2 like:

      12:01:00,641 WARN  [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=32, bqual_length=36, tx_uid=0:ffff7f000001:23447aae:68ee36b9:124d, node_name=5026, branch_uid
      =0:ffff7f000001:23447aae:68ee36b9:1250, subordinatenodename=null, eis_name=java:/LocalJmsXA NodeId:7d43c294-a8f2-11f0-af76-fa163efd220b >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ Artemis/2.42.0, jndiName: java:/LocalJmsXA NodeId:7d43c294-a8f2-11f0-af76-fa163efd220b com.arjuna.ats.internal.jta.resource
      s.arjunacore.XAResourceRecord@498a716a >
      12:01:00,642 WARN  [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=32, bqual_length=36, tx_uid=0:ffff7f000001:23447aae:68ee36b9:124d, node_name=5026, branch_uid
      =0:ffff7f000001:23447aae:68ee36b9:12de, subordinatenodename=null, eis_name=java:/JmsXA NodeId:8374d7cc-a8f2-11f0-ae53-fa163efd220b >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ Artemis/2.0, jndiName: java:/JmsXA NodeId:8374d7cc-a8f2-11f0-ae53-fa163efd220b com.arjuna.ats.internal.jta.resources.arjunacore.
      XAResourceRecord@32a4244d >
      12:01:00,642 WARN  [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=32, bqual_length=36, tx_uid=0:ffff7f000001:23447aae:68ee36b9:124d, node_name=5026, branch_uid=0:ffff7f000001:23447aae:68ee36b9:1250, subordinatenodename=null, eis_name=java:/LocalJmsXA No
      deId:7d43c294-a8f2-11f0-af76-fa163efd220b >
      12:01:00,642 WARN  [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=32, bqual_length=36, tx_uid=0:ffff7f000001:23447aae:68ee36b9:124d, node_name=5026, branch_uid=0:ffff7f000001:23447aae:68ee36b9:12de, subordinatenodename=null, eis_name=java:/JmsXA NodeId:8374d7cc-a8f2-11f0-ae53-fa163efd220b >
      

      It indicates that recovery manager knows that there are unfinished transactions but cannot do anything as it's not able to find correct XAResource.

              ehugonne1@redhat.com Emmanuel Hugonnet
              mnovak1@redhat.com Miroslav Novak
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: