Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-1329

org.infinispan.loaders.decorators.AsyncStore leads to OutOfMemoryExceptions

    XMLWordPrintable

Details

    Description

      When I configure

       
      <async
          enabled="true"
          threadPoolSize="3"/>
      

      the implementation creates three threads names "CoalescedAsyncStore" per cache store.

      When adding a large amount of data to the cache (for example indexing a lot of Lucene documents), the application will stop working with a lot of OutOfMemory exceptions.

      This is caused by the implementation of org.infinispan.loaders.decorators.AsyncStore that uses an unbounded LinkedBlockingQueue.
      A lot of worker threads push a huge amount of modifications into the queues, but the underlying cache store is not able to handle throughput.

      As a quick solution change

         private final LinkedBlockingQueue<Modification> changesDeque = new LinkedBlockingQueue<Modification>();
      

      to

         private final LinkedBlockingQueue<Modification> changesDeque = new LinkedBlockingQueue<Modification>(512);
      

      and add a new configuration option to the async element, that specifies the capacity of the Linked Blocking Queue.

      Attachments

        Activity

          People

            manik_jira Manik Surtani (Inactive)
            snazy0 Robert Stupp (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: