Uploaded image for project: 'Kogito'
  1. Kogito
  2. KOGITO-2207

Kogito App holds all process instances in memory

    XMLWordPrintable

Details

    • Hide

      1. Run the process-quarkus-example with -Xmx3G heap setting, i.e.

      java -Xmx3G -jar target/process-quarkus-example-runner.jar
      

      2. Create a large number of process instances, e.g. 40 000 by issuing the POST request to the path /orders with the body:

      {
              "approver" : "john",
              "order" : {
                "orderNumber" : "12345",
                "shipped" : false
              }
      }
      

      3. Send a GET request to the path /orders a couple of times, e.g. using a browser. An OutOfMemoryError will occur after a few tries.

      Show
      1. Run the process-quarkus-example with -Xmx3G heap setting, i.e. java -Xmx3G -jar target/process-quarkus-example-runner.jar 2. Create a large number of process instances, e.g. 40 000 by issuing the POST request to the path /orders with the body: { "approver" : "john" , "order" : { "orderNumber" : "12345" , "shipped" : false } } 3. Send a GET request to the path /orders a couple of times, e.g. using a browser. An OutOfMemoryError will occur after a few tries.
    • 2020 Week 25-27 (from Jun 15), 2020 Week 28-30 (from Jul 6), 2020 Week 31-33 (from Jul 27)

    Description

      When running the process-quarkus-example Kogito App with Infinispan persistence, I have observed the following situation.

      When an order process instance is created, it also creates an orderItems subprocess which will create a human task with an assigned work item. Moreover, the parent process instance order is registered as a listener via its LightSignalManager on a SignalManagerHub because it needs to listen for completion of its orderItems subprocess. This means that all process instances are kept in memory even though they are not needed. Moreover, all HumanTaskWorkItemImpl objects, which represent human tasks belonging to the orderItems subprocess, are also kept in memory and are referenced by the LightWorkItemManager. Each HumanTaskWorkItemImpl points to the concrete NodeInstance which in turn points to the ProcessInstance, effectively holding again all process instances in memory. Finally, process instances are also referenced by the DefaultProcessInstanceManager, so they are kept in memory from this point as well.

      Another issue is that after hitting /orders endpoint the second time, there are additional 40 000 process instances added in memory even though they had already been loaded by the previous request. This is the reason why the OutOfMemoryError occurs after hitting the same endpoint a couple of times. It just runs out of free memory as more and more process instances are loaded to memory.

      After the Kogito App is restarted, it doesn't hold any process instances, but after another /orders request, all processes are loaded to memory again.

      I have analyzed this using the Eclipse Memory Analyzer. I have heap dumps obtained at various points of debugging of this behaviour, so I can provide them if needed. But they can be generated pretty quickly by following the Steps to Reproduce.

      Attachments

        Activity

          People

            cnicolai@redhat.com Cristiano Nicolai
            mmacik@redhat.com Marian Macik
            Marian Macik Marian Macik
            Marian Macik Marian Macik
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: