Uploaded image for project: 'JBoss Enterprise Application Platform 4 and 5'
  1. JBoss Enterprise Application Platform 4 and 5
  2. JBPAPP-7673

Consumer does not failover when hardware crash of server occur

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Blocker Blocker
    • EAP_EWP 5.1.2 CR2
    • EAP_EWP 5.1.2 CR1
    • HornetQ
    • None
    • Not Required

      There is problem with failover of consumer when server to which it's connected is killed.

      Test scenario:
      1. Start EAP server A on physical server A(messaging-06) and EAP server B on physical server B(messaging-05) in collocated topology - servers contain backup for each other
      2. Start producer and consumer connected to server A - clients are active for the whole duration of the test
      3. Physically crash server A (power off) -> consumer won't failover to backup server A on server B

      There is guide how to reproducer test scenario with disconnected journal in messaging lab. I've attached reproducer.zip and logs.zip with server logs and consumer thread dumps during hw crash.

      How to reproduce:

      1. Log to any messaging machine and under Hudson user set JAVA_HOME
      ssh messaging-xx.jbm.lab.bos.redhat.com (replace xx by number like 04)
      sudo su - hudson
      export JAVA_HOME=/qa/tools/opt/amd64/jdk1.6.0_24
      export PATH=$JAVA_HOME/bin:$PATH
      cd ... (some working directory)

      2. Download and unzip reproducer.zip

      3. Prepare EAP server with profiles for server A and B in collocated topology by using command:
      sh prepare-servers.sh

      Now each of the following steps need open a new console, ssh to messaging machine, switch under hudson user, set JAVA_HOME and change to working directory of the reproducer - "messaging-xx" is hostname with EAP server A and "messaging-xy" is hostname with EAP server B - reuse commands from step 1.
      4. Start server A - "sh start-server1.sh messaging-xx" (replace xx by number of messaging machine)
      5. On another messaging machine start server B - "sh start-server2.sh messaging-xy" (replace xy by number of messaging machine)
      6. Start producer connected to server A - "sh start_producer.sh messaging-xx" (connected messaging-xx - queue/Inqueue)
      7. Start consumer connected to server A - "sh start_consumer.sh messaging-xx" (connected messaging-xx - queue/Inqueue)
      8. Now power off server A
      a). Open web browser and go to: https://messaging-xx-rsa.jbm.lab.bos.redhat.com/private/userlogin.ssi (replace xx by number of server A)
      b). Log In using credentials I've sent you some time ago
      c). Power off the server -> "Power/Restart" -> "Shut down OS and then Power Off Server" -> "OK"
      d). Log off

      I saw following unexpected behavior:
      a) After step 8. only producer managed to failover to backup but consumer not (thread dump from consumer is in logs.zip)

      Let me know if there is a problem.

        1. jmsClient.zip
          127 kB
        2. logs.zip
          308 kB
        3. reproducer.zip
          9.04 MB

            mnovak1@redhat.com Miroslav Novak
            mnovak1@redhat.com Miroslav Novak
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: