Uploaded image for project: 'JBoss A-MQ'
  1. JBoss A-MQ
  2. ENTMQ-812

java.lang.OutOfMemoryError: GC overhead limit exceeded when there is a problem with NFSv4 KahaDB back-end storage

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Major Major
    • None
    • None
    • broker, kahadb, message-store
    • None

      I have three A-MQ brokers inside Jboss Fuse 6.1 fabric, one master and two slaves. Back-end storage for kahadb is NFSv4. Now we have issues on the storage side and the nodes are loosing the connection. In dmesg:

      NFS: server my.nfs.server.com error: fileid changed
      fsid 0:13: expected fileid 0x32, got 0xc000c

      At some point (usually 1-2 days) the master broker is giving:
      2014-09-25 13:44:52,455 | WARN | 925772421-209484 | nio | tty.io.nio.SelectChannelEndPoint 697 | 92 - org.eclipse.jetty.aggregate.jetty-all-server - 8.1.14.v20131031 | handle failed
      java.lang.OutOfMemoryError: GC overhead limit exceeded

      And the slave cannot pick-up because the master don't release the NFS lock. Then I have to manually stop the master container.
      Note that this server has very low load, 50 messages per day or something. And nagios checks to jolokia every few minutes.

              Unassigned Unassigned
              pntdevops-vvasilev Vladimir Vasilev
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: