INFO | Establishing network connection from vm://sydlab?async=false&network=true to failover:(tcp://10.64.132.143:61616,tcp://10.64.132.151:61616)
 INFO | Connector vm://sydlab started
 INFO | Network Connector DiscoveryNetworkConnector:sydlab4-nc:BrokerService[sydlab] started
 INFO | Apache ActiveMQ 5.9.0.redhat-610164 (sydlab, ID:rhel6-x64-noel-1.syd.lab.redhat.com-57443-1385005107848-0:1) started
 INFO | For help or more information please see: http://activemq.apache.org
 WARN | Store limit is 102400 mb (current store usage is 400 mb). The data directory: /opt/jboss-amq/test/node2/apache-activemq-5.9.0.redhat-610164/bin/../data/leveldb/amq only has 37521 mb of usable space - resetting to maximum available disk space: 37921 mb
ERROR | Temporary Store limit is 51200 mb, whilst the temporary data directory: /opt/jboss-amq/test/node2/apache-activemq-5.9.0.redhat-610164/data/sydlab/tmp_storage only has 37521 mb of usable space - resetting to maximum available 37521 mb.
 INFO | Successfully connected to tcp://10.64.132.143:61616
 INFO | Network connection between vm://sydlab#0 and tcp://10.64.132.143:61616 (lab41) has been established.
 INFO | Slave has connected: 368b9536-7e93-4223-809e-a87742561796
 INFO | Slave has now caught up: 368b9536-7e93-4223-809e-a87742561796
 WARN | Too many cluster members are connected.  Expected at most 3 members but there are 4 connected.
 INFO | Slave has connected: b1b6250b-5209-4246-8482-c19f0fc8528e
 INFO | Slave has now caught up: b1b6250b-5209-4246-8482-c19f0fc8528e
 WARN | Unexpected session error: java.io.IOException: Connection reset by peer
 INFO | Slave has disconnected: 2a3a61de-5388-47ac-a5f6-70d17489425e
 WARN | Unexpected session error: java.io.IOException: Connection reset by peer
 INFO | Slave has disconnected: b1b6250b-5209-4246-8482-c19f0fc8528e
 WARN | Unexpected session error: java.io.IOException: Connection reset by peer
 INFO | Slave has disconnected: 368b9536-7e93-4223-809e-a87742561796
 INFO | Not enough cluster members connected to elect a master.
 WARN | listeners are taking too long to process the events
 WARN | listeners are taking too long to process the events
 INFO | Not enough cluster members connected to elect a master.
 INFO | Demoted to slave
 WARN | Transport Connection to: vm://sydlab#0 failed: java.io.IOException: Store has been stopped
 INFO | Apache ActiveMQ 5.9.0.redhat-610164 (sydlab, ID:rhel6-x64-noel-1.syd.lab.redhat.com-57443-1385005107848-0:1) is shutting down
 INFO | Connector vm://sydlab stopped
 INFO | sydlab bridge to lab41 stopped
 INFO | Network Connector DiscoveryNetworkConnector:sydlab4-nc:BrokerService[sydlab] stopped
 INFO | Stopped LevelDB[/opt/jboss-amq/test/node2/apache-activemq-5.9.0.redhat-610164/bin/../data/leveldb/amq]
 INFO | Master stopped
 INFO | Not enough cluster members connected to elect a master.
 INFO | Not enough cluster members connected to elect a master.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
2013-11-21 14:41:37
Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.12-b01 mixed mode):

"ActiveMQ InactivityMonitor Worker" daemon prio=10 tid=0x00007f4378004800 nid=0x52f1 waiting on condition [0x00007f43b55a4000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000ffe28900> (a java.util.concurrent.SynchronousQueue$TransferStack)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
	at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424)
	at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
	at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
	at java.lang.Thread.run(Thread.java:662)

"ActiveMQ InactivityMonitor WriteCheckTimer" daemon prio=10 tid=0x00007f43c0843800 nid=0x52e4 in Object.wait() [0x00007f43b4c9b000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.util.TimerThread.mainLoop(Timer.java:509)
	- locked <0x00000000ffe17f00> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:462)

"ActiveMQ InactivityMonitor ReadCheckTimer" daemon prio=10 tid=0x00007f43c0812800 nid=0x52e3 in Object.wait() [0x00007f43b4d9c000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.util.TimerThread.mainLoop(Timer.java:509)
	- locked <0x00000000ffe28460> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:462)

"ActiveMQ BrokerService[sydlab] Task-3" daemon prio=10 tid=0x00007f4384039800 nid=0x52e2 waiting on condition [0x00007f43b4e9d000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000ffccac20> (a java.util.concurrent.CountDownLatch$Sync)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303)
	at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:253)
	at org.apache.activemq.broker.TransportConnection.stop(TransportConnection.java:983)
	at org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:41)
	at org.apache.activemq.broker.TransportConnector.stop(TransportConnector.java:286)
	at org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:41)
	at org.apache.activemq.broker.BrokerService.stopAllConnectors(BrokerService.java:2026)
	at org.apache.activemq.broker.BrokerService.stop(BrokerService.java:727)
	at org.apache.activemq.xbean.XBeanBrokerService.stop(XBeanBrokerService.java:122)
	at org.apache.activemq.leveldb.replicated.ElectingLevelDBStore$$anonfun$stop_master$2.apply$mcV$sp(ElectingLevelDBStore.scala:253)
	at org.fusesource.hawtdispatch.package$$anon$4.run(hawtdispatch.scala:357)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)

"ActiveMQ Transport: tcp:///10.64.132.143:46428@61616" daemon prio=10 tid=0x00007f4384096800 nid=0x52e1 waiting on condition [0x00007f43b509f000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000f3177c88> (a java.util.concurrent.CountDownLatch$Sync)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
	at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207)
	at org.apache.activemq.leveldb.CountDownFuture.get(DBManager.scala:122)
	at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:870)
	at org.apache.activemq.broker.region.Queue.send(Queue.java:700)
	at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:394)
	at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:441)
	at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:283)
	at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:147)
	at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
	at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:307)
	at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:147)
	at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:152)
	at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:467)
	at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:751)
	at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
	at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:149)
	at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
	at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
	at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:270)
	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:214)
	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
	at java.lang.Thread.run(Thread.java:662)

"leveldb" prio=10 tid=0x00007f438c003800 nid=0x52cd runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"ActiveMQ BrokerService[sydlab] Task-2" daemon prio=10 tid=0x00007f437c0ba000 nid=0x52ca waiting on condition [0x00007f43b6a0d000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000ffcd6ba8> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
	at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:807)
	at org.apache.activemq.broker.TransportConnection$4.run(TransportConnection.java:1035)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)

"hawtdispatch timer" daemon prio=10 tid=0x00007f437c013000 nid=0x52c8 in Object.wait() [0x00007f43b6c0f000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at org.fusesource.hawtdispatch.internal.TimerThread.run(TimerThread.java:186)
	- locked <0x00000000c024e6e0> (a java.lang.Object)

"hawtdispatch-DEFAULT-2" daemon prio=10 tid=0x00007f437c06c800 nid=0x52c7 runnable [0x00007f43b6d10000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
	- locked <0x00000000c02535d8> (a sun.nio.ch.Util$2)
	- locked <0x00000000c02535c8> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000000c024f378> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
	at org.fusesource.hawtdispatch.internal.NioManager$SelectStrategy.select(NioManager.java:52)
	at org.fusesource.hawtdispatch.internal.NioManager.select(NioManager.java:182)
	at org.fusesource.hawtdispatch.internal.pool.SimplePool.park(SimplePool.java:130)
	at org.fusesource.hawtdispatch.internal.pool.SimpleThread.run(SimpleThread.java:75)

"hawtdispatch-DEFAULT-1" daemon prio=10 tid=0x00007f437c012000 nid=0x52c6 runnable [0x00007f43b6e11000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
	- locked <0x00000000c024ee70> (a sun.nio.ch.Util$2)
	- locked <0x00000000c024ee60> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000000c024ec38> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
	at org.fusesource.hawtdispatch.internal.NioManager$SelectStrategy.select(NioManager.java:52)
	at org.fusesource.hawtdispatch.internal.NioManager.select(NioManager.java:182)
	at org.fusesource.hawtdispatch.internal.pool.SimplePool.park(SimplePool.java:130)
	at org.fusesource.hawtdispatch.internal.pool.SimpleThread.run(SimpleThread.java:75)

"main-EventThread" daemon prio=10 tid=0x00007f43c07bb800 nid=0x529a waiting on condition [0x00007f43b6f12000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000c0280778> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)

"main-SendThread(rhel6-x64-noel-3:2183)" daemon prio=10 tid=0x00007f43c07ba800 nid=0x5299 runnable [0x00007f43b7013000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
	- locked <0x00000000c02788e8> (a sun.nio.ch.Util$2)
	- locked <0x00000000c02788d8> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000000c0278448> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)

"ZooKeeper state change dispatcher thread" daemon prio=10 tid=0x00007f43c0869800 nid=0x5298 waiting on condition [0x00007f43b7114000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000c026cd88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
	at org.apache.activemq.leveldb.replicated.groups.ZKClient$StateChangeDispatcher.run(ZKClient.java:347)

"RMI TCP Accept-0" daemon prio=10 tid=0x00007f43c0148000 nid=0x5295 runnable [0x00007f43b76d9000]
   java.lang.Thread.State: RUNNABLE
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
	- locked <0x00000000c009b150> (a java.net.SocksSocketImpl)
	at java.net.ServerSocket.implAccept(ServerSocket.java:462)
	at java.net.ServerSocket.accept(ServerSocket.java:430)
	at sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:34)
	at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
	at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
	at java.lang.Thread.run(Thread.java:662)

"Low Memory Detector" daemon prio=10 tid=0x00007f43c00b4800 nid=0x5294 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" daemon prio=10 tid=0x00007f43c00b2800 nid=0x5293 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" daemon prio=10 tid=0x00007f43c00af800 nid=0x5292 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x00007f43c00ad800 nid=0x5291 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=10 tid=0x00007f43c0091000 nid=0x5290 in Object.wait() [0x00007f43b7dfc000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000000c013ec38> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
	- locked <0x00000000c013ec38> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x00007f43c008f000 nid=0x528f in Object.wait() [0x00007f43b7efd000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000000c00000a0> (a java.lang.ref.Reference$Lock)
	at java.lang.Object.wait(Object.java:485)
	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
	- locked <0x00000000c00000a0> (a java.lang.ref.Reference$Lock)

"main" prio=10 tid=0x00007f43c0007800 nid=0x528b waiting on condition [0x00007f43c423b000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000ffcdcb68> (a java.util.concurrent.CountDownLatch$Sync)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
	at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207)
	at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:118)
	at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
	at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
	at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
	at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
	at org.apache.activemq.console.Main.main(Main.java:115)

"VM Thread" prio=10 tid=0x00007f43c0088800 nid=0x528e runnable

"GC task thread#0 (ParallelGC)" prio=10 tid=0x00007f43c001a800 nid=0x528c runnable

"GC task thread#1 (ParallelGC)" prio=10 tid=0x00007f43c001c800 nid=0x528d runnable

"VM Periodic Task Thread" prio=10 tid=0x00007f43c0157000 nid=0x5296 waiting on condition

JNI global references: 1161

Heap
 PSYoungGen      total 346048K, used 155527K [0x00000000eaab0000, 0x0000000100000000, 0x0000000100000000)
  eden space 342592K, 44% used [0x00000000eaab0000,0x00000000f40acdf8,0x00000000ff940000)
  from space 3456K, 56% used [0x00000000ffca0000,0x00000000ffe85040,0x0000000100000000)
  to   space 3456K, 0% used [0x00000000ff940000,0x00000000ff940000,0x00000000ffca0000)
 PSOldGen        total 699072K, used 2708K [0x00000000c0000000, 0x00000000eaab0000, 0x00000000eaab0000)
  object space 699072K, 0% used [0x00000000c0000000,0x00000000c02a5318,0x00000000eaab0000)
 PSPermGen       total 37376K, used 37246K [0x00000000bae00000, 0x00000000bd280000, 0x00000000c0000000)
  object space 37376K, 99% used [0x00000000bae00000,0x00000000bd25f8e0,0x00000000bd280000)

 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.
 INFO | The connection to 'tcp://10.64.132.143:46428' is taking a long time to shutdown.

<<<<<<<<<<<<<<<<<<<<<<<<< Stopped master and restarted all nodes to reproduce the issue
The same output occurred but when I restarted an AMQ node to reform the quorum the following exception occurred



 INFO | The connection to 'tcp://10.64.132.143:55732' is taking a long time to shutdown.
 INFO | Not enough cluster members have reported their update positions yet.
 INFO | Promoted to master
 WARN | [/fabric/registry/clusters/fusemq-replication-elections/default/00000000449] [ZooKeeperTreeTracker@34469729] [Thread[main-EventThread,5,main]] Error in treeWatcher (ignored)
java.lang.AssertionError: assertion failed
        at scala.Predef$.assert(Predef.scala:89)
        at org.apache.activemq.leveldb.replicated.ElectingLevelDBStore.start_master(ElectingLevelDBStore.scala:222)
        at org.apache.activemq.leveldb.replicated.MasterElector$change_listener$.changed(MasterElector.scala:200)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$$anonfun$fireChanged$1$$anonfun$apply$mcV$sp$3.apply(ChangeListener.scala:89)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$$anonfun$fireChanged$1$$anonfun$apply$mcV$sp$3.apply(ChangeListener.scala:88)
        at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
        at scala.collection.immutable.List.foreach(List.scala:45)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$$anonfun$fireChanged$1.apply$mcV$sp(ChangeListener.scala:88)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$$anonfun$fireChanged$1.apply(ChangeListener.scala:88)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$$anonfun$fireChanged$1.apply(ChangeListener.scala:88)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$class.check_elapsed_time(ChangeListener.scala:97)
        at org.apache.activemq.leveldb.replicated.groups.ClusteredSingletonWatcher.check_elapsed_time(ClusteredSingleton.scala:82)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$class.fireChanged(ChangeListener.scala:87)
        at org.apache.activemq.leveldb.replicated.groups.ClusteredSingletonWatcher.fireChanged(ClusteredSingleton.scala:82)
        at org.apache.activemq.leveldb.replicated.groups.ClusteredSingletonWatcher.changed_decoded(ClusteredSingleton.scala:161)
        at org.apache.activemq.leveldb.replicated.groups.ClusteredSingletonWatcher$$anon$2.changed(ClusteredSingleton.scala:107)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$$anonfun$fireChanged$1$$anonfun$apply$mcV$sp$3.apply(ChangeListener.scala:89)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$$anonfun$fireChanged$1$$anonfun$apply$mcV$sp$3.apply(ChangeListener.scala:88)
        at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
        at scala.collection.immutable.List.foreach(List.scala:45)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$$anonfun$fireChanged$1.apply$mcV$sp(ChangeListener.scala:88)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$$anonfun$fireChanged$1.apply(ChangeListener.scala:88)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$$anonfun$fireChanged$1.apply(ChangeListener.scala:88)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$class.check_elapsed_time(ChangeListener.scala:97)
        at org.apache.activemq.leveldb.replicated.groups.ZooKeeperGroup.check_elapsed_time(ZooKeeperGroup.scala:73)
        at org.apache.activemq.leveldb.replicated.groups.ChangeListenerSupport$class.fireChanged(ChangeListener.scala:87)
        at org.apache.activemq.leveldb.replicated.groups.ZooKeeperGroup.fireChanged(ZooKeeperGroup.scala:73)
        at org.apache.activemq.leveldb.replicated.groups.ZooKeeperGroup.org$apache$activemq$leveldb$replicated$groups$ZooKeeperGroup$$fire_cluster_change(ZooKeeperGroup.scala:165)
        at org.apache.activemq.leveldb.replicated.groups.ZooKeeperGroup$$anon$1.onEvents(ZooKeeperGroup.scala:88)
        at org.linkedin.zookeeper.tracker.ZooKeeperTreeTracker.raiseEvents(ZooKeeperTreeTracker.java:402)
        at org.linkedin.zookeeper.tracker.ZooKeeperTreeTracker.access$800(ZooKeeperTreeTracker.java:50)
        at org.linkedin.zookeeper.tracker.ZooKeeperTreeTracker$TreeWatcher.process(ZooKeeperTreeTracker.java:114)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
 INFO | The connection to 'tcp://10.64.132.143:55732' is taking a long time to shutdown.