-
Bug
-
Resolution: Done
-
Major
-
None
-
None
-
None
-
- start cassandra node
- start hawkular services
- generate load (POST requests to hawkular metrics)
Hawkular services JVM is spending all the time on Full GC under heavy load so some requests fail with 500 response because of timeout issues.
Performance test is hitting hawkular metrics with POST requests. There is no other activity.
I didn't search for exact commit but it was definitely caused by something after Oct 5. Build from Oct 5 passed.
Here is thread dump - http://web.bc.jonqe.lab.eng.bos.redhat.com/jstack-issue.out
Here is heap dump - http://web.bc.jonqe.lab.eng.bos.redhat.com/haw_ser_dump_issue.bin
All logs including GC log are attached.
I used just 512MB for heap in this test to reproduce it faster but it was visible even with 4GB for heap.