Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-8547

Artemis live does not shutdown if NFS or GFS2 is disconnected

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Blocker
    • 7.1.0.CR2
    • 7.1.0.DR10, 7.1.0.DR18, 7.1.0.ER3
    • ActiveMQ
    • None
    • AMQ Sprint 3

    Description

      Customer Impact: If live server has journal located on NFSv4 shared store then if it looses connection then it does not shutdown itself and does not allow clients to failover to backup. Clients will crash or retry connection to live server. As HA does not take, the service is unavailable.

      Test Scenario:

      • Start 2 EAP servers (live/backup pair) in dedicated HA topology with shared store on NFSv4.
      • Start producer/consumer sending/receiving messages to/from queue on live server
      • Disconnect cable which connected live server with NFSv4 server (connection to backup server and client connections were not disconnected as they're going though another cable/NIC)
      • Wait for live to shutdown itself on IO critical error

      Expected results: Backup will activate and clients failover to backup. Number of send and received messages is the same.

      Actual result:
      When cable between live and NFSv4 server is disconnected then live does not shutdown itself on critical IO error and no failover happens. Clients hangs as well. Attaching thread dumps from servers and clients.

      NFSv4 server is configured to export directory in /etc/exports:

      /mnt/shared *(rw,sync)
      

      NFS clients mount this export in /etc/fstab:

      10.40.4.25:/mnt/shared         /mnt/shared-mount  nfs4     minorversion=0,rw,nosuid,nodev,sync,lock,intr,noac,soft,lookupcache=none,sloppy,actimeo=10,retry=0,timeo=5,retrans=2  0 0
      192.168.10.1:/mnt/shared         /mnt/shared  nfs4         minorversion=0,rw,nosuid,nodev,sync,lock,intr,noac,soft,lookupcache=none,sloppy,actimeo=10,retry=0,timeo=5,retrans=2  0 0
      

      Attaching server.log from live and backup.

      Attachments

        1. dump_live_dr18
          103 kB
        2. live-log.zip
          5.02 MB
        3. standalone-full-ha-backup.xml
          25 kB
        4. standalone-full-ha-live.xml
          25 kB
        5. thread_dumps_with_fixes.zip
          20 kB
        6. thread-dump-backup.txt
          96 kB
        7. thread-dump-live.txt
          149 kB
        8. thread-dump-producer.txt
          29 kB
        9. thread-dump-receiver.txt
          29 kB

        Issue Links

          Activity

            People

              mtaylor1@redhat.com Martyn Taylor (Inactive)
              mnovak1@redhat.com Miroslav Novak
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: