Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-19931

[CLUSTERING] ISPN000472: Cache manager is stopping

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 35.0.0.Beta1
    • Clustering
    • None
    • ---
    • ---

      We have a 4 nodes cluster configured as in the following to connect to an external PostreSQL database:

      embed-server --server-config=standalone-ha.xml
      /subsystem=jgroups/channel=ee:write-attribute(name=stack,value=tcp)
      data-source add --name=testDS --jndi-name=java:jboss/datasources/testDS --driver-name=postgresql-connector.jar --connection-url=jdbc:postgresql://host:5432/dbname --enabled=true --jta=true --use-java-context=true --transaction-isolation=TRANSACTION_READ_COMMITTED --min-pool-size=10 --max-pool-size=100 --pool-prefill=true --user-name=username --password=pass.1234
      

      We deploy second-level-cache.ear on each node; this is a Hibernate application configured with:

      <property name="hibernate.cache.use_second_level_cache" value="true"/>
      

      All nodes a shutdown and restarted in a sequence;

      When node 4 is stopped we observe the following error:

      2024-11-11 08:50:17,962 ERROR [org.infinispan.CLUSTER] (thread-26,ejb,wildfly4) ISPN000474: Error processing request 3590@wildfly3: org.infinispan.commons.IllegalLifecycleStateException: ISPN000472: Cache manager is stopping
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.GlobalMarshaller.getExternalizer(GlobalMarshaller.java:429)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.GlobalMarshaller.readWithExternalizer(GlobalMarshaller.java:726)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.GlobalMarshaller.readNonNullableObject(GlobalMarshaller.java:708)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.GlobalMarshaller.readNullableObject(GlobalMarshaller.java:357)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.BytesObjectInput.readObject(BytesObjectInput.java:32)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.commands.CommandInvocationId.readFrom(CommandInvocationId.java:55)
      	at org.infinispan.hibernate-cache@14.0.32.Final//org.infinispan.hibernate.cache.commons.util.LockOwner.readFrom(LockOwner.java:35)
      	at org.infinispan.hibernate-cache@14.0.32.Final//org.infinispan.hibernate.cache.commons.util.BeginInvalidationCommand.readFrom(BeginInvalidationCommand.java:45)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.exts.ReplicableCommandExternalizer.readCommandParameters(ReplicableCommandExternalizer.java:116)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.exts.ReplicableCommandExternalizer.readObject(ReplicableCommandExternalizer.java:105)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.exts.ReplicableCommandExternalizer.readObject(ReplicableCommandExternalizer.java:67)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.GlobalMarshaller.readWithExternalizer(GlobalMarshaller.java:727)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.GlobalMarshaller.readNonNullableObject(GlobalMarshaller.java:708)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.GlobalMarshaller.readNullableObject(GlobalMarshaller.java:357)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.BytesObjectInput.readObject(BytesObjectInput.java:32)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.commands.remote.SingleRpcCommand.readFrom(SingleRpcCommand.java:56)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.exts.ReplicableCommandExternalizer.readCommandParameters(ReplicableCommandExternalizer.java:116)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:166)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:85)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.GlobalMarshaller.readWithExternalizer(GlobalMarshaller.java:727)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.GlobalMarshaller.readNonNullableObject(GlobalMarshaller.java:708)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.GlobalMarshaller.readNullableObject(GlobalMarshaller.java:357)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.GlobalMarshaller.objectFromObjectInput(GlobalMarshaller.java:191)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.marshall.core.GlobalMarshaller.objectFromByteBuffer(GlobalMarshaller.java:220)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport.processRequest(JGroupsTransport.java:1528)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1468)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.lambda$up$1(JGroupsTransport.java:1677)
      	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
      	at org.infinispan.core@14.0.32.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.up(JGroupsTransport.java:1669)
      	at org.jgroups@5.2.29.Final//org.jgroups.JChannel.up(JChannel.java:750)
      	at org.jgroups@5.2.29.Final//org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:161)
      	at org.jgroups@5.2.29.Final//org.jgroups.stack.Protocol.up(Protocol.java:373)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.FORK.up(FORK.java:174)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.FRAG2.up(FRAG2.java:161)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.FlowControl.up(FlowControl.java:319)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.FlowControl.up(FlowControl.java:319)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.pbcast.GMS.up(GMS.java:862)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:246)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.UNICAST3.up(UNICAST3.java:501)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.pbcast.NAKACK2.deliverBatch(NAKACK2.java:1033)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.pbcast.NAKACK2.handleMessageBatch(NAKACK2.java:936)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:769)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.VERIFY_SUSPECT2.up(VERIFY_SUSPECT2.java:119)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.FailureDetection.up(FailureDetection.java:193)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.FD_SOCK2.up(FD_SOCK2.java:203)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.MERGE3.up(MERGE3.java:288)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.Discovery.up(Discovery.java:314)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.RED.up(RED.java:119)
      	at org.jgroups@5.2.29.Final//org.jgroups.protocols.TP.passBatchUp(TP.java:1210)
      	at org.jgroups@5.2.29.Final//org.jgroups.util.SubmitToThreadPool$BatchHandler.passBatchUp(SubmitToThreadPool.java:161)
      	at org.jgroups@5.2.29.Final//org.jgroups.util.SubmitToThreadPool$BatchHandler.run(SubmitToThreadPool.java:157)
      	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
      	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
      	at org.wildfly.clustering.context@1.1.2.Final//org.wildfly.clustering.context.ContextualExecutor$1.execute(ContextualExecutor.java:180)
      	at org.wildfly.clustering.context@1.1.2.Final//org.wildfly.clustering.context.ContextualExecutor.execute(ContextualExecutor.java:31)
      	at org.wildfly.clustering.context@1.1.2.Final//org.wildfly.clustering.context.Contextualizer$2$1.run(Contextualizer.java:220)
      	at java.base/java.lang.Thread.run(Thread.java:840)
      

      Complete configuration and logs in wildfly.zip;

        1. second-level-cache.ear
          1.15 MB
          Tommaso Borgato

              pferraro@redhat.com Paul Ferraro
              tborgato@redhat.com Tommaso Borgato
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: