Uploaded image for project: 'HornetQ'
  1. HornetQ
  2. HORNETQ-659

Memory leak in cluster implementation

    Details

      Description

      We have a serious issue in our production system, which makes that HornetQ goes out of memory after aprox. 12 hours. The heap size is already 4GB and the issue still occurs. I was able to reproduce the issue locally by creating a simple requestor program and a replyer program (which more or less simulates usage in production). The requestor puts a message in the HornetQ cluster on one queue and waits for a reply on another queue. After about 20000 messages (size between 5K and 25K), one of the HornetQ cluster nodes goes out of mem.

      We tracked in down using visualvm: there appears to be a memory leak in org.hornetq.core.server.cluster.impl.RemoteQueueBindingImpl. In the code below you can see that filters are added in a hashset, but they can never be removed because the hashset can never find the object on the remove call. This is because add(FilterImpl) and remove(String) is called and their hashvalues will never match.

      public synchronized void addConsumer(final SimpleString filterString) throws Exception
      {
      if (filterString != null)
      {
      // There can actually be many consumers on the same queue with the same filter, so we need to maintain a ref
      // count

      Integer i = filterCounts.get(filterString);

      if (i == null)

      { filterCounts.put(filterString, 0); filters.add(FilterImpl.createFilter(filterString)); }

      else

      { filterCounts.put(filterString, i + 1); }

      }

      consumerCount++;
      }

      public synchronized void removeConsumer(final SimpleString filterString) throws Exception
      {
      if (filterString != null)
      {
      Integer i = filterCounts.get(filterString);

      if (i != null)
      {
      int ii = i - 1;

      if (ii == 0)

      { filterCounts.remove(filterString); filters.remove(filterString); }

      else

      { filterCounts.put(filterString, ii); }

      }
      }

      consumerCount--;
      }

      In attach also a picture of visualvm with the memory consumtion of the filters attribute.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                clebert.suconic Clebert Suconic
                Reporter:
                jeroen.v Jeroen Verellen
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: