Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-19374

[CLUSTERING] java.lang.NullPointerException 'because "v" is null' when node leaves the cluster

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Blocker Blocker
    • None
    • 33.0.0.Beta1
    • Clustering
    • None
    • ---
    • ---

      We have a 4 nodes cluster;

      The 'because "v" is null' error pops out when one node leaves the cluster, on one or more other nodes in the cluster;

      For example, when WildFly on node1 is shut down, we observe the following ERROR on nodes 2 and 3:

      2024-05-31 08:13:13,189 INFO  [org.infinispan.CLUSTER] (thread-19,ejb,wildfly2) [Context=org.infinispan.CONFIG] ISPN100009: Advancing to rebalance phase READ_ALL_WRITE_ALL, topology id 18
      2024-05-31 08:13:13,191 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (thread-15,ejb,wildfly2) ISPN000136: Error executing command PrepareCommand on Cache 'clusterbench-ee10.ear/clusterbench-ee10-ejb.jar/default', writing keys [InfinispanBeanMetaDataKey(UUIDSessionID [4f6533cf-e0df-42c7-8db9-733011fc8bb1]), InfinispanBeanGroupKey(UUIDSessionID [4f6533cf-e0df-42c7-8db9-733011fc8bb1])]: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from wildfly3, see cause for remote stack trace
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.transport.ResponseCollectors.wrapRemoteException(ResponseCollectors.java:26)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.transport.impl.MapResponseCollector.addException(MapResponseCollector.java:65)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.transport.impl.MapResponseCollector$IgnoreLeavers.addException(MapResponseCollector.java:103)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.transport.ValidResponseCollector.addResponse(ValidResponseCollector.java:29)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.transport.impl.MultiTargetRequest.onResponse(MultiTargetRequest.java:98)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.transport.impl.RequestRepository.addResponse(RequestRepository.java:51)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport.processResponse(JGroupsTransport.java:1570)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1471)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.lambda$up$1(JGroupsTransport.java:1677)
      	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.up(JGroupsTransport.java:1669)
      	at org.jgroups@5.2.25.Final//org.jgroups.JChannel.up(JChannel.java:750)
      	at org.jgroups@5.2.25.Final//org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:161)
      	at org.jgroups@5.2.25.Final//org.jgroups.stack.Protocol.up(Protocol.java:373)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.FORK.up(FORK.java:174)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.FRAG2.up(FRAG2.java:161)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.FlowControl.up(FlowControl.java:319)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.FlowControl.up(FlowControl.java:319)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.pbcast.GMS.up(GMS.java:862)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:246)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.UNICAST3.deliverBatch(UNICAST3.java:1111)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.UNICAST3.handleBatchReceived(UNICAST3.java:885)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.UNICAST3.up(UNICAST3.java:552)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:722)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.VERIFY_SUSPECT2.up(VERIFY_SUSPECT2.java:119)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.FailureDetection.up(FailureDetection.java:193)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.FD_SOCK2.up(FD_SOCK2.java:203)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.MERGE3.up(MERGE3.java:288)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.Discovery.up(Discovery.java:314)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.RED.up(RED.java:119)
      	at org.jgroups@5.2.25.Final//org.jgroups.protocols.TP.passBatchUp(TP.java:1210)
      	at org.jgroups@5.2.25.Final//org.jgroups.util.SubmitToThreadPool$BatchHandler.passBatchUp(SubmitToThreadPool.java:161)
      	at org.jgroups@5.2.25.Final//org.jgroups.util.SubmitToThreadPool$BatchHandler.run(SubmitToThreadPool.java:157)
      	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
      	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
      	at org.wildfly.clustering.context@33.0.0.Beta1-202405301447-2cfc39a8//org.wildfly.clustering.context.ContextReferenceExecutor.execute(ContextReferenceExecutor.java:32)
      	at org.wildfly.clustering.context@33.0.0.Beta1-202405301447-2cfc39a8//org.wildfly.clustering.context.ContextualExecutor$1.run(ContextualExecutor.java:61)
      	at java.base/java.lang.Thread.run(Thread.java:840)
      Caused by: java.lang.NullPointerException: Cannot invoke "org.infinispan.container.entries.MVCCEntry.isRemoved()" because "v" is null
      	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
      	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
      	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.exts.ThrowableExternalizer.newThrowableInstance(ThrowableExternalizer.java:286)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.exts.ThrowableExternalizer.readGenericThrowable(ThrowableExternalizer.java:264)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.exts.ThrowableExternalizer.readObject(ThrowableExternalizer.java:239)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.exts.ThrowableExternalizer.readObject(ThrowableExternalizer.java:44)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.core.GlobalMarshaller.readWithExternalizer(GlobalMarshaller.java:727)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.core.GlobalMarshaller.readNonNullableObject(GlobalMarshaller.java:708)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.core.GlobalMarshaller.readNullableObject(GlobalMarshaller.java:357)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.core.BytesObjectInput.readObject(BytesObjectInput.java:32)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.responses.ExceptionResponse$Externalizer.readObject(ExceptionResponse.java:49)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.responses.ExceptionResponse$Externalizer.readObject(ExceptionResponse.java:41)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.core.GlobalMarshaller.readWithExternalizer(GlobalMarshaller.java:727)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.core.GlobalMarshaller.readNonNullableObject(GlobalMarshaller.java:708)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.core.GlobalMarshaller.readNullableObject(GlobalMarshaller.java:357)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.core.GlobalMarshaller.objectFromObjectInput(GlobalMarshaller.java:191)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.marshall.core.GlobalMarshaller.objectFromByteBuffer(GlobalMarshaller.java:220)
      	at org.infinispan.core@14.0.28.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport.processResponse(JGroupsTransport.java:1562)
      	... 31 more
      

      The client is not affected though;

            pferraro@redhat.com Paul Ferraro
            tborgato@redhat.com Tommaso Borgato
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: