Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-4249

When a node left and rejoined the cluster, the authentication failed to sync.

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Critical
    • Resolution: Done
    • Affects Version/s: 8.1.0.Final, 8.2.0.Final, 9.0.0.Alpha1
    • Fix Version/s: 9.0.0.Beta1
    • Component/s: Clustering
    • Labels:
      None
    • Steps to Reproduce:
      Hide

      1. Enabled SSO
      In standalone-ha.xml, enable SSO.
      <single-sign-on/>
      2. Start 2 nodes:
      $JBOSS_HOME/bin/standalone.sh -c standalone-ha.xml -b=10.0.0.120 -u 230.0.0.4 -Djboss.node.name=node3 -Djboss.socket.binding.port-offset=100
      $JBOSS_HOME/bin/standalone.sh -c standalone-ha.xml -b=10.0.0.120 -u 230.0.0.4 -Djboss.node.name=node4 -Djboss.socket.binding.port-offset=200
      3. Login and shut down one of the nodes and then restart the node.

      Show
      1. Enabled SSO In standalone-ha.xml, enable SSO. <single-sign-on/> 2. Start 2 nodes: $JBOSS_HOME/bin/standalone.sh -c standalone-ha.xml -b=10.0.0.120 -u 230.0.0.4 -Djboss.node.name=node3 -Djboss.socket.binding.port-offset=100 $JBOSS_HOME/bin/standalone.sh -c standalone-ha.xml -b=10.0.0.120 -u 230.0.0.4 -Djboss.node.name=node4 -Djboss.socket.binding.port-offset=200 3. Login and shut down one of the nodes and then restart the node.
    • Affects:
      User Experience

      Description

      Hi there,

      When I try to use cluster features, I got an issue.

      With SSO enabled, when a node left and rejoined the cluster, the authentication failed to sync.

      I enabled SSO in standalone-ha.xml
      <single-sign-on/>

      Node 3:
      $JBOSS_HOME/bin/standalone.sh -c standalone-ha.xml -b=10.0.0.120 -u 230.0.0.4 -Djboss.node.name=node3 -Djboss.socket.binding.port-offset=100

      Node 4:
      $JBOSS_HOME/bin/standalone.sh -c standalone-ha.xml -b=10.0.0.120 -u 230.0.0.4 -Djboss.node.name=node4 -Djboss.socket.binding.port-offset=200

      I first started both nodes and logged in. Then I shut down node 3, and refresh the page. Everything worked fine until now. After this, I started node 3 again but I found I was logged out. When I shut down the node3, it worked again.

      I also tried different versions of WildFly, 8.1, 8.2 and 9.0 and it failed in all versions.

      The log from node 4 below happens sometimes but NOT always.
      14:56:44,874 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-18,shared=udp) ISPN000094: Received new cluster view: [node4/web|3] (2) [node4/web, node3/web]
      14:56:45,117 ERROR [org.infinispan.statetransfer.OutboundTransferTask] (transport-thread-12) Failed to send entries to node node3/web : ISPN000217: Received exception from node3/web, see cause for remote stack trace: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from node3/web, see cause for remote stack trace
      at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:41)
      at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:362)
      at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:167)
      at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:521)
      at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:281)
      at org.infinispan.statetransfer.OutboundTransferTask.sendEntries(OutboundTransferTask.java:233)
      at org.infinispan.statetransfer.OutboundTransferTask.run(OutboundTransferTask.java:174)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_45]
      at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45]
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_45]
      at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45]
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
      at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
      Caused by: org.infinispan.commons.CacheException: Problems invoking command.
      at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:221)
      at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:460)
      at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:377)
      at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247)
      at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:667)
      at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130)
      at org.jgroups.JChannel.up(JChannel.java:708)
      at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1015)
      at org.jgroups.protocols.RSVP.up(RSVP.java:171)
      at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
      at org.jgroups.protocols.FlowControl.up(FlowControl.java:381)
      at org.jgroups.protocols.FlowControl.up(FlowControl.java:370)
      at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1010)
      at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
      at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:695)
      at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:382)
      at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:600)
      at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:147)
      at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:185)
      at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:301)
      at org.jgroups.protocols.MERGE3.up(MERGE3.java:303)
      at org.jgroups.protocols.Discovery.up(Discovery.java:379)
      at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2622)
      at org.jgroups.protocols.TP.passMessageUp(TP.java:1412)
      at org.jgroups.protocols.TP$MyHandler.run(TP.java:1598)
      ... 3 more
      Caused by: java.io.InvalidClassException: com.britensw.nyx.core.security.NyxPrincipal; Module load failed
      at org.jboss.marshalling.ModularClassResolver.resolveClass(ModularClassResolver.java:104) [jboss-marshalling-1.4.8.Final.jar:1.4.8.Final]
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:949)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1256)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224)
      at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1746)
      at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1659)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1286)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224)
      at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1746)
      at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1659)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1286)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41) [jboss-marshalling-1.4.8.Final.jar:1.4.8.Final]
      at org.wildfly.clustering.web.infinispan.sso.coarse.CoarseAuthenticationEntryExternalizer.readObject(CoarseAuthenticationEntryExternalizer.java:57)
      at org.wildfly.clustering.web.infinispan.sso.coarse.CoarseAuthenticationEntryExternalizer.readObject(CoarseAuthenticationEntryExternalizer.java:36)
      at org.infinispan.marshall.core.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:389)
      at org.infinispan.marshall.core.ExternalizerTable.readObject(ExternalizerTable.java:205)
      at org.infinispan.marshall.core.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:152)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:355)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41) [jboss-marshalling-1.4.8.Final.jar:1.4.8.Final]
      at org.infinispan.container.entries.ImmortalCacheEntry$Externalizer.readObject(ImmortalCacheEntry.java:152)
      at org.infinispan.container.entries.ImmortalCacheEntry$Externalizer.readObject(ImmortalCacheEntry.java:142)
      at org.infinispan.marshall.core.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:389)
      at org.infinispan.marshall.core.ExternalizerTable.readObject(ExternalizerTable.java:205)
      at org.infinispan.marshall.core.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:152)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:355)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41) [jboss-marshalling-1.4.8.Final.jar:1.4.8.Final]
      at org.infinispan.marshall.exts.ListExternalizer.readObject(ListExternalizer.java:62)
      at org.infinispan.marshall.exts.ListExternalizer.readObject(ListExternalizer.java:26)
      at org.infinispan.marshall.core.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:389)
      at org.infinispan.marshall.core.ExternalizerTable.readObject(ExternalizerTable.java:205)
      at org.infinispan.marshall.core.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:152)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:355)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41) [jboss-marshalling-1.4.8.Final.jar:1.4.8.Final]
      at org.infinispan.statetransfer.StateChunk$Externalizer.readObject(StateChunk.java:88)
      at org.infinispan.statetransfer.StateChunk$Externalizer.readObject(StateChunk.java:65)
      at org.infinispan.marshall.core.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:389)
      at org.infinispan.marshall.core.ExternalizerTable.readObject(ExternalizerTable.java:205)
      at org.infinispan.marshall.core.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:152)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:355)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41) [jboss-marshalling-1.4.8.Final.jar:1.4.8.Final]
      at org.infinispan.marshall.exts.ListExternalizer.readObject(ListExternalizer.java:62)
      at org.infinispan.marshall.exts.ListExternalizer.readObject(ListExternalizer.java:26)
      at org.infinispan.marshall.core.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:389)
      at org.infinispan.marshall.core.ExternalizerTable.readObject(ExternalizerTable.java:205)
      at org.infinispan.marshall.core.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:152)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:355)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41) [jboss-marshalling-1.4.8.Final.jar:1.4.8.Final]
      at org.infinispan.marshall.exts.ReplicableCommandExternalizer.readParameters(ReplicableCommandExternalizer.java:100)
      at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:146)
      at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:59)
      at org.infinispan.marshall.core.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:389)
      at org.infinispan.marshall.core.ExternalizerTable.readObject(ExternalizerTable.java:205)
      at org.infinispan.marshall.core.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:152)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:355)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41) [jboss-marshalling-1.4.8.Final.jar:1.4.8.Final]
      at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:136)
      at org.infinispan.marshall.core.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:101)
      at org.infinispan.commons.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:80)
      at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:28)
      at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:206)
      ... 27 more
      Caused by: org.jboss.modules.ModuleNotFoundException: deployment.nyx.war:main
      at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:240) [jboss-modules.jar:1.3.4.Final]
      at org.jboss.marshalling.ModularClassResolver.resolveClass(ModularClassResolver.java:102) [jboss-marshalling-1.4.8.Final.jar:1.4.8.Final]
      ... 96 more
      Caused by: an exception which occurred:
      in field principal
      in object of type org.wildfly.extension.undertow.security.AccountImpl
      in field account
      in object of type io.undertow.security.api.AuthenticatedSessionManager$AuthenticatedSession

      14:56:49,679 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,shared=udp) ISPN000094: Received new cluster view: [node4/ejb|3] (2) [node4/ejb, node3/ejb]
      14:56:51,515 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-15,shared=udp) ISPN000094: Received new cluster view: [node4/hibernate|3] (2) [node4/hibernate, node3/hibernate]

      Apache configuration:
      <<mod_cluster.conf>>
      LoadModule proxy_module modules/mod_proxy.so
      LoadModule proxy_http_module modules/mod_proxy_http.so
      LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
      LoadModule slotmem_module modules/mod_slotmem.so
      LoadModule manager_module modules/mod_manager.so
      LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
      LoadModule advertise_module modules/mod_advertise.so

      MemManagerFile /var/cache/httpd

      <<cluster.conf>>
      Listen 10001
      <VirtualHost *:10001>
      ServerAdmin root@example.com
      ServerName localhost

      <Directory />
      Require all granted
      </Directory>

      <Location /mod_cluster_manager>
      SetHandler mod_cluster-manager
      Require all granted
      </Location>

      AdvertiseFrequency 5
      KeepAliveTimeout 60
      MaxKeepAliveRequests 0
      AllowDisplay On

      ManagerBalancerName mycluster
      ServerAdvertise On
      EnableMCPMReceive

      ErrorLog /var/log/httpd/error_log
      TransferLog /var/log/httpd/access_log
      </VirtualHost>

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                pferraro Paul Ferraro
                Reporter:
                stwang46 Jixing Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: