class WritableServiceBasedNamingStore installs a new MSC service (instance of BinderService) and then waits for that MSC service to boot - by creating a new listener of type BindListener (an inner class) and calling listener.await() (line 72 in WritableServiceBasedNamingStore.java).
i have a @Singleton @Startup EJB which, in its @PostConstruct method calls (new InitialContext().bind(...)), and hits this code.
that @PostConstruct method was itself invoked by an MSC thread. the end result looks like this:
"MSC service thread 1-2" prio=6 tid=0x000000000e146000 nid=0x960 in Object.wait() [0x000000000f6fd000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000007f652e400> (a org.jboss.as.naming.WritableServiceBasedNamingStore$BindListener)
at java.lang.Object.wait(Object.java:503)
at org.jboss.as.naming.WritableServiceBasedNamingStore$BindListener.await(WritableServiceBasedNamingStore.java:153) - locked <0x00000007f652e400> (a org.jboss.as.naming.WritableServiceBasedNamingStore$BindListener)
at org.jboss.as.naming.WritableServiceBasedNamingStore.bind(WritableServiceBasedNamingStore.java:72)
at org.jboss.as.naming.NamingContext.bind(NamingContext.java:221)
at org.jboss.as.naming.InitialContext.bind(InitialContext.java:172)
at org.jboss.as.naming.NamingContext.bind(NamingContext.java:230)
at javax.naming.InitialContext.bind(Unknown Source)
[application code ommitted]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.as.ee.component.ManagedReferenceFieldInjectionInterceptorFactory$ManagedReferenceFieldInjectionInterceptor.processInvocation(ManagedReferenceFieldInjectionInterceptorFactory.java:112)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.as.ee.component.ManagedReferenceFieldInjectionInterceptorFactory$ManagedReferenceFieldInjectionInterceptor.processInvocation(ManagedReferenceFieldInjectionInterceptorFactory.java:112)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.as.ee.component.ManagedReferenceFieldInjectionInterceptorFactory$ManagedReferenceFieldInjectionInterceptor.processInvocation(ManagedReferenceFieldInjectionInterceptorFactory.java:112)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.as.ee.component.ManagedReferenceFieldInjectionInterceptorFactory$ManagedReferenceFieldInjectionInterceptor.processInvocation(ManagedReferenceFieldInjectionInterceptorFactory.java:112)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228)
at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333)
at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161)
at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85)
at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) - locked <0x000000078b4108e8> (a java.lang.Object)
at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130)
at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44)
at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
notice that this stack is during ComponentStartService.start(), which is in itself an MSC service. this is against the Service contract which states services should never block (or use the async api)
this also has real world implications -
the specific vm this was run on has a single (virtual) cpu, which means the MSC pool size is 2, one of which is now waiting.
if two of my @Singletons happen to coincide the server deadlocks and will never finish booting (the thread dump excerpt above was taken just from such a situation)
- is related to
-
AS7-4530 BasicComponent#waitForComponentStart() can result in deadlock if there is a deployment failure
- Resolved