• Icon: Bug Bug
    • Resolution: Done
    • Icon: Critical Critical
    • 8.0 Update 2
    • 7.4.12.GA, 8.0.0.GA-CR1
    • Clustering, MP Metrics
    • None

      ./standalone.sh
      cp jsf_reproducer.war deployments
      # this redeploys application in infinite loop
      watch -n 1 touch jsf_reproducer.war.dodeploy
      
      # show leak
      wget   https://github.com/check-leak/check-leak/releases/download/0.11/check-leak-0.11.jar
      java -jar check-leak-0.11.jar remote --pid <PID_OF_EAP> --report /tmp/report --sleep 5000
      

      Tool https://github.com/check-leak/check-leak works on principle of reporting new maximum of instances per class. After some time report stabilize to this output

      *******************************************************************************************************************************
      Executing...
      Processing histogram
      |          3005296 bytes (+208)|          57779 instances (+4)|[Ljava.lang.Thread; (java.base@17.0.7)
      |          2773680 bytes (+192)|          57785 instances (+4)|java.lang.ThreadGroup (java.base@17.0.7)
      |            693264 bytes (+48)|          14443 instances (+1)|org.infinispan.factories.threads.BlockingThreadFactory$ISPNBlockingThreadGroup
      |            693264 bytes (+48)|          14443 instances (+1)|org.infinispan.factories.threads.NonBlockingThreadFactory$ISPNNonBlockingThreadGroup
      |            346680 bytes (+24)|          14445 instances (+1)|org.wildfly.extension.metrics.MetricCollector
      |            346680 bytes (+24)|          14445 instances (+1)|org.wildfly.extension.metrics.MetricRegistration
      *******************************************************************************************************************************
      

      This mean those classes reach new occurence maximum each time application is redeployed. Even running GC cant stop that observation. Maximum is continuosly rising.
      Overal number of instances match number of redeploys. When I stop redeploying app new maxima are not reported. Increasing trend can be seen also in this diagram of Old Gen Memory Heap area.

      I can observe same behaviour in EAP74, so this is not regression compared to EAP 74.

            [JBEAP-25513] (8.0.z) Memory leak on app redeploy

            As all verified issues should change to closed status, closing them with the bulk update.

            Michaela Osmerova added a comment - As all verified issues should change to closed status, closing them with the bulk update.

            Radoslav Husar added a comment - - edited

            Lowering to Critical (root cause is known, incomplete fix for ISPN-15147 in Infinispan).

            Radoslav Husar added a comment - - edited Lowering to Critical (root cause is known, incomplete fix for ISPN-15147 in Infinispan).

            thofman Yes, I believe we want to fix it also in 74. I have created clone JBEAP-25573 to track that work. In both streams I set default priority as Blocker to be triaged properly as it deals with memory leak.

            Martin Choma added a comment - thofman Yes, I believe we want to fix it also in 74. I have created clone JBEAP-25573 to track that work. In both streams I set default priority as Blocker to be triaged properly as it deals with memory leak.

            So I believe that ISPN-15147 and JBEAP-25545 should be enough to resolve this fully. Both sub-component issues are now ready (PR up or merged).

            Tomas Hofman added a comment - So I believe that ISPN-15147 and JBEAP-25545 should be enough to resolve this fully. Both sub-component issues are now ready (PR up or merged).

            mchoma@redhat.com just to make sure - this is intended to be a blocker for EAP 8.0.0.GA?

            Tomas Hofman added a comment - mchoma@redhat.com just to make sure - this is intended to be a blocker for EAP 8.0.0.GA?

            Thanks remerson@redhat.com, I happy with you resolving the ISPN-15147. I'm going to handle this ticket as a composition of ISPN-15147 and a different fix in EAP metrics, I will resolve it myself when both are fixed.

            Tomas Hofman added a comment - Thanks remerson@redhat.com , I happy with you resolving the ISPN-15147 . I'm going to handle this ticket as a composition of ISPN-15147 and a different fix in EAP metrics, I will resolve it myself when both are fixed.

            mchoma@redhat.com do we want to have this fixed also in EAP 7.4?

            Tomas Hofman added a comment - mchoma@redhat.com do we want to have this fixed also in EAP 7.4?

            thofman The Infinispan PR has been merged. I haven't marked this Jira as resolved as I'm not sure what the correct version should be.

            Ryan Emerson added a comment - thofman The Infinispan PR has been merged. I haven't marked this Jira as resolved as I'm not sure what the correct version should be.

            I think there are possibly multiple leaks detected (unless they stem from a single source).

            1. Classes around org.wildfly.extension.metrics.MetricRegistration,
            2. Infinispan related classes (org.infinispan.factories.threads.BlockingThreadFactory$ISPNBlockingThreadGroup),
            3. [Ljava.lang.Thread and java.lang.ThreadGroup possibly relalted to infinispan above.

            Tomas Hofman added a comment - I think there are possibly multiple leaks detected (unless they stem from a single source). 1. Classes around org.wildfly.extension.metrics.MetricRegistration, 2. Infinispan related classes (org.infinispan.factories.threads.BlockingThreadFactory$ISPNBlockingThreadGroup), 3. [Ljava.lang.Thread and java.lang.ThreadGroup possibly relalted to infinispan above.

              thofman Tomas Hofman
              mchoma@redhat.com Martin Choma
              Martin Choma Martin Choma
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

                Created:
                Updated:
                Resolved: