Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-14547

Session offloading broken with IllegalStateException: ISPN006016: Factory 'org.infinispan.server.hotrod.HotRodServer$ToEmptyBytesKeyValueFilterConverter' not found in server

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Won't Do
    • Icon: Critical Critical
    • 7.2.0.CD12
    • 7.2.0.CD12
    • Clustering
    • None

      Our session offloading tests (provided by JDG team) fail with

      10:59:52,492 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 21) MSC000001: Failed to start service org.wildfly.clustering.infinispan.cache.web.jws-session-dist-async.war: org.jboss.msc.service.StartException in service org.wildfly.clustering.infinispan.cache.web.jws-session-dist-async.war: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.preload() on object of type PersistenceManagerImpl
      	at org.wildfly.clustering.service.FunctionalValueService.start(FunctionalValueService.java:70)
      	at org.wildfly.clustering.service.AsynchronousServiceBuilder.lambda$start$0(AsynchronousServiceBuilder.java:99)
      	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
      	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
      	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
      	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)
      	at java.lang.Thread.run(Thread.java:748)
      	at org.jboss.threads.JBossThread.run(JBossThread.java:485)
      Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.preload() on object of type PersistenceManagerImpl
      	at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:83)
      	at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
      	at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
      	at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:165)
      	at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
      	at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:635)
      	at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:624)
      	at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:549)
      	at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:232)
      	at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:987)
      	at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
      	at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:653)
      	at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:598)
      	at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:462)
      	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:448)
      	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:434)
      	at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:60)
      	at org.wildfly.clustering.infinispan.spi.service.CacheBuilder.lambda$build$0(CacheBuilder.java:83)
      	at org.wildfly.clustering.service.SuppliedValueService.lambda$new$0(SuppliedValueService.java:42)
      	at org.wildfly.clustering.service.FunctionalValueService.start(FunctionalValueService.java:68)
      	... 7 more
      Caused by: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for messageId=3 returned server error (status=0x85): java.lang.IllegalStateException: ISPN006016: Factory 'org.infinispan.server.hotrod.HotRodServer$ToEmptyBytesKeyValueFilterConverter' not found in server
      	at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:363)
      	at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:152)
      	at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:138)
      	at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:60)
      	at org.infinispan.client.hotrod.impl.operations.IterationStartOperation.executeOperation(IterationStartOperation.java:72)
      	at org.infinispan.client.hotrod.impl.operations.IterationStartOperation.executeOperation(IterationStartOperation.java:21)
      	at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:56)
      	at org.infinispan.client.hotrod.impl.iteration.RemoteCloseableIterator.startInternal(RemoteCloseableIterator.java:127)
      	at org.infinispan.client.hotrod.impl.iteration.RemoteCloseableIterator.start(RemoteCloseableIterator.java:140)
      	at org.infinispan.client.hotrod.impl.RemoteCacheImpl.retrieveEntries(RemoteCacheImpl.java:163)
      	at org.infinispan.client.hotrod.impl.RemoteCacheImpl.retrieveEntries(RemoteCacheImpl.java:169)
      	at org.infinispan.client.hotrod.impl.RemoteCacheImpl.retrieveEntries(RemoteCacheImpl.java:174)
      	at org.infinispan.client.hotrod.impl.RemoteCacheImpl$KeySet.iterator(RemoteCacheImpl.java:562)
      	at org.infinispan.persistence.remote.RemoteStore.process(RemoteStore.java:161)
      	at org.infinispan.persistence.manager.PersistenceManagerImpl.preload(PersistenceManagerImpl.java:272)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
      	... 26 more
      

      Link to the test console output: https://eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/jdg-func-session-externalization-eap/23/CONFIG=DOMAIN,container=eap,jdk=oracle-java-1.8,label_exp=RHEL7,strategy=COARSE/consoleFull

      This points to ISPN-8719 issue.

      After initial investigation, we concluded with help of vjuranek@redhat.com that the problem is the following: EAP 7.2.0.CD12.CR2 uses ISPN 9.1.7 JDG 7.1.2 uses ISPN 8.4.2 Meaning, EAP uses newer HotRod protocol version when communicating with JDG server.

      Update: The protocol versions turned out to be the same, but the ISPN-8719 issue is valid.

      Summary: The original issue, as presented on PM list was: EAP 7.2.0.CD12.CR2 cannot be configured to externalize sessions into JDG cluster. This has proven not to be correct. EAP 7.2.0.CD12.CR2 can be configured to externalize sessions into JDG cluster, the only non-working configuration is, when "preload" is set to "true". New Jira JBEAP-14607 has been opened for the "preload=true" issue.

              rhn-engineering-rhusar Radoslav Husar
              mvinkler1@redhat.com Michal Vinkler
              Tommaso Borgato Tommaso Borgato
              Tommaso Borgato Tommaso Borgato
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: