Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-24661

Clustering - New Error in EJB + Servlet Undeploy sceanrio

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • None
    • 8.0.0.Beta-CR1
    • Clustering
    • None
    • False
    • None
    • False

      We have 4 nodes cluster and we have deployed a clustered application using EJB + Servlet; the application is un-deployed and then re-deployed from each node of the cluster in sequence;

      After the application is un-deployed from Node-1 we observe the following errors which are basically endless and go on an on until the test ends and the nodes are finally shut down (complete logs in wildfly.zip ):

      Node-1:

      2023-03-23 10:46:19,040 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 89) WFLYUT0022: Unregistered web context: '/clusterbench' from server 'default-server'
      2023-03-23 10:46:19,041 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 90) WFLYUT0022: Unregistered web context: '/clusterbench-passivating' from server 'default-server'
      2023-03-23 10:46:19,044 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 94) WFLYUT0022: Unregistered web context: '/clusterbench-granular' from server 'default-server'
      2023-03-23 10:46:19,071 ERROR [io.undertow.request] (default task-2) UT005023: Exception handling request to /clusterbench/ejbservlet: org.jboss.as.ejb3.component.EJBComponentUnavailableException: WFLYEJB0421: Invocation cannot proceed as component is shutting down
          at org.jboss.as.ejb3@8.0.0.GA-redhat-20230321//org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:59)
          at org.jboss.invocation@2.0.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
          at org.jboss.as.ejb3@8.0.0.GA-redhat-20230321//org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
          at org.jboss.invocation@2.0.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
          at org.jboss.as.ee@8.0.0.GA-redhat-20230321//org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
          at org.jboss.invocation@2.0.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
      ...
      

      Node-2:

      2023-03-23 10:47:54,473 INFO  [org.infinispan.LIFECYCLE] (thread-42,ejb,wildfly2) [Context=clusterbench-ee10.ear/clusterbench-ee10-ejb.jar/default] ISPN100010: Finished rebalance with members [wildfly2, wildfly3, wildfly4, wildfly1], topology id 19
      2023-03-23 10:49:31,111 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 23) WFLYUT0022: Unregistered web context: '/web-logger' from server 'default-server'
      2023-03-23 10:49:31,192 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-2) WFLYSRV0028: Stopped deployment web-logger.war (runtime-name: web-logger.war) in 85ms
      2023-03-23 10:49:31,297 INFO  [org.jboss.as.repository] (management-handler-thread - 1) WFLYDR0002: Content removed from location /tmp/tests-clustering/jboss-eap-2/standalone/data/content/4a/d32c946992dc12f8358cd103e4f04e5fc09cc6/content
      2023-03-23 10:49:31,298 INFO  [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0009: Undeployed "web-logger.war" (runtime-name: "web-logger.war")
      2023-03-23 10:49:31,448 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 89) WFLYUT0022: Unregistered web context: '/clusterbench-passivating' from server 'default-server'
      2023-03-23 10:49:31,449 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 90) WFLYUT0022: Unregistered web context: '/clusterbench' from server 'default-server'
      2023-03-23 10:49:31,475 ERROR [io.undertow.request] (default task-12) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException
          at org.jboss.as.weld.ejb@8.0.0.GA-redhat-20230321//org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133)
          at org.jboss.weld.core@5.1.0.Final-redhat-00001//org.jboss.weld.module.ejb.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:126)
          at org.jboss.weld.core@5.1.0.Final-redhat-00001//org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)
      

      Node-3:

      2023-03-23 10:49:41,715 INFO  [org.infinispan.LIFECYCLE] (thread-45,ejb,wildfly3) [Context=default-server] ISPN100002: Starting rebalance with members [wildfly3, wildfly4, wildfly1], phase READ_OLD_WRITE_ALL, topology id 24
      2023-03-23 10:49:41,716 INFO  [org.infinispan.LIFECYCLE] (non-blocking-thread--p8-t3) [Context=default-server] ISPN100010: Finished rebalance with members [wildfly3, wildfly4, wildfly1], topology id 24
      2023-03-23 10:49:45,808 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p15-t1) ISPN000136: Error executing command GetKeyValueCommand on Cache 'clusterbench-ee10.ear.clusterbench-ee10-web.war', writing keys []: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 127453 from wildfly4 after 10 seconds
          at org.infinispan@14.0.7.Final//org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:171)
          at org.infinispan@14.0.7.Final//org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:88)
          at org.infinispan@14.0.7.Final//org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22)
          at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          at java.base/java.lang.Thread.run(Thread.java:834)
      

      Node-4:

      topology id 24
      2023-03-23 10:49:41,714 INFO  [org.infinispan.LIFECYCLE] (thread-59,ejb,wildfly4) [Context=default-server] ISPN100002: Starting rebalance with members [wildfly3, wildfly4, wildfly1], phase READ_OLD_WRITE_ALL, topology id 24
      2023-03-23 10:49:41,715 INFO  [org.infinispan.LIFECYCLE] (non-blocking-thread--p4-t2) [Context=default-server] ISPN100010: Finished rebalance with members [wildfly3, wildfly4, wildfly1], topology id 24
      2023-03-23 10:49:46,790 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p16-t1) ISPN000136: Error executing command GetKeyValueCommand on Cache 'clusterbench-ee10.ear.clusterbench-ee10-web.war', writing keys []: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key SessionCreationMetaDataKey(wHvSa_Pbl8i4e33vq9JkOd6qJfj-5fDgOTJYD5cY) and requestor GlobalTransaction{id=158455, addr=wildfly4, remote=false, xid=null, internalId=-1}. Lock is held by GlobalTransaction{id=158298, addr=wildfly2, remote=true, xid=null, internalId=-1} (pending)
          at org.infinispan@14.0.7.Final//org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager.timeout(DefaultPendingLockManager.java:148)
          at org.infinispan@14.0.7.Final//org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager$PendingLockPromiseImpl.onRelease(DefaultPendingLockManager.java:328)
          at org.infinispan@14.0.7.Final//org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager$PendingLockPromiseImpl.call(DefaultPendingLockManager.java:293)
          at org.infinispan@14.0.7.Final//org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager$PendingLockPromiseImpl.call(DefaultPendingLockManager.java:246)
          at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          at java.base/java.lang.Thread.run(Thread.java:834)
      

      The client is affected by this error;

        1. wildfly(1).zip
          671 kB
        2. wildfly.zip
          482 kB

              Unassigned Unassigned
              tborgato@redhat.com Tommaso Borgato
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: