Details
-
Bug
-
Resolution: Cannot Reproduce
-
Minor
-
None
-
7.4.0.Beta
-
None
-
False
-
False
-
Undefined
Description
The error is present in EAP Clustering tests where HTTP session data is replicated;
We spotted the error in 2 kind of tests:
- tests with replicated cache where fail-over is introduced by shutting down nodes
- tests with replicated cache where fail-over is introduced by shutting down nodes and replication level is granular
- tests with replicated cache and shared-sessions where fail-over is introduced by jvmkill
- tests with replicated SFSB
In the first kind of tests the nodes are configured as follows:
embed-server --server-config=standalone-ha.xml /subsystem=jgroups/channel=ee:write-attribute(name=stack,value=tcp) /subsystem=infinispan/cache-container=web/replicated-cache=testRepl:add() /subsystem=infinispan/cache-container=web/replicated-cache=testRepl/component=locking:write-attribute(name=isolation, value=REPEATABLE_READ) /subsystem=infinispan/cache-container=web/replicated-cache=testRepl/component=transaction:write-attribute(name=mode, value=BATCH) /subsystem=infinispan/cache-container=web/replicated-cache=testRepl/store=file:add() /subsystem=infinispan/cache-container=web:write-attribute(name=default-cache, value=testRepl)
In the second kind of tests the nodes are configured as follows:
embed-server --server-config=standalone-ha.xml /subsystem=jgroups/channel=ee:write-attribute(name=stack,value=tcp) /subsystem=infinispan/cache-container=web/replicated-cache=testRepl:add() /subsystem=infinispan/cache-container=web/replicated-cache=testRepl/component=locking:write-attribute(name=isolation, value=REPEATABLE_READ) /subsystem=infinispan/cache-container=web/replicated-cache=testRepl/component=transaction:write-attribute(name=mode, value=BATCH) /subsystem=infinispan/cache-container=web/replicated-cache=testRepl/store=file:add() /subsystem=infinispan/cache-container=web:write-attribute(name=default-cache, value=testRepl)
In the fourth kind of tests the nodes are configured as follows:
embed-server --server-config=standalone-ha.xml /subsystem=jgroups/channel=ee:write-attribute(name=stack,value=tcp) /subsystem=infinispan/cache-container=web/replicated-cache=testReplWeb:add() /subsystem=infinispan/cache-container=web/replicated-cache=testReplWeb/component=locking:write-attribute(name=isolation, value=REPEATABLE_READ) /subsystem=infinispan/cache-container=web/replicated-cache=testReplWeb/component=transaction:write-attribute(name=mode, value=BATCH) /subsystem=infinispan/cache-container=web/replicated-cache=testReplWeb/store=file:add() /subsystem=infinispan/cache-container=web:write-attribute(name=default-cache, value=testReplWeb) /subsystem=infinispan/cache-container=ejb/replicated-cache=testReplEjb:add() /subsystem=infinispan/cache-container=ejb/replicated-cache=testReplEjb/component=locking:write-attribute(name=isolation, value=REPEATABLE_READ) /subsystem=infinispan/cache-container=ejb/replicated-cache=testReplEjb/component=transaction:write-attribute(name=mode, value=BATCH) /subsystem=infinispan/cache-container=ejb/replicated-cache=testReplEjb/store=file:add() /subsystem=infinispan/cache-container=ejb:write-attribute(name=default-cache, value=testReplEjb)
The error happens after the nodes are restarted:
2021-02-15 11:41:19,763 FATAL [org.jgroups.protocols.TCP] (Connection.Receiver [10.0.144.84:7600 - 10.0.146.24:51588]-9,ejb,wildfly1) wildfly1: thread pool is full (max=100, active=200); thread dump (dumped once, until thread_dump is reset):
Reference Handler:
at java.lang.ref.Reference.waitForReferencePendingList(Reference.java:-2)
at java.lang.ref.Reference.processPendingReferences(Reference.java:241)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)
Finalizer:
at java.lang.Object.wait(Object.java:-2)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)
Signal Dispatcher:
Common-Cleaner:
at java.lang.Object.wait(Object.java:-2)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
at jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
at java.lang.Thread.run(Thread.java:834)
at jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)
Reference Reaper:
at java.lang.Object.wait(Object.java:-2)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
at org.jboss.modules.ref.References$ReaperThread.run(References.java:64)
MSC service thread 1-1:
at jdk.internal.misc.Unsafe.park(Unsafe.java:-2)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1421)
at java.lang.Thread.run(Thread.java:834)
MSC service thread 1-2:
at jdk.internal.misc.Unsafe.park(Unsafe.java:-2)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1421)
at java.lang.Thread.run(Thread.java:834)
etc...
The client is not impacted by the error;
Find complete logs attached;
Complete runs here:
- eap-7.x-clustering-http-session-shutdown-repl-haproxy#56
- eap-7.x-clustering-http-session-granular-repl-shutdown-haproxy#46
- eap-7.x-clustering-http-session-shutdown-repl#119
- eap-7.x-clustering-http-shared-sessions-repl-jvmkill#33
- eap-7.x-clustering-ejb-ejbservlet-jvmkill-repl#43
- eap-7.x-clustering-ejb-ejbservlet-jvmkill-repl#44
- eap-7.x-clustering-ejb-ejbservlet-shutdown-repl#51
- eap-7.x-clustering-http-session-shutdown-repl-granular#4
Attachments
Issue Links
- relates to
-
-
- Verified
-