We had the situation that our JGroups thread pool was full. Because of huge load on our servers, the thread dump took 30 seconds, which caused the node to leave the cluster which caused a lot of other trouble.
[2024-05-27T10:26:29.129+0200][4553.843s][info][safepoint] Safepoint "ThreadDump", Time since last: 100445138 ns, Reaching safepoint: 149578 ns, Cleanup: 5814824 ns, At safepoint: 30810585003 ns, Total: 30816549405 ns
We now set
thread_pool.thread_dumps_threshold="0"
to avoid taking a thread dump at all. But we do not get any message about this situation.
It would be better to always log this line in case of RejectedExecutionException
tp.getLog().error("failure submitting task to thread pool", t);
If no thread dump is taken, nothing gets logged. So the second time the thread pool is full, nothing gets logged. (if you didn't reset the thread dumps counter)