Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-14547

Session offloading broken with IllegalStateException: ISPN006016: Factory 'org.infinispan.server.hotrod.HotRodServer$ToEmptyBytesKeyValueFilterConverter' not found in server

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Do
    • Critical
    • 7.2.0.CD12
    • 7.2.0.CD12
    • Clustering
    • None

    Description

      Our session offloading tests (provided by JDG team) fail with 

      10:59:52,492 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 21) MSC000001: Failed to start service org.wildfly.clustering.infinispan.cache.web.jws-session-dist-async.war: org.jboss.msc.service.StartException in service org.wildfly.clustering.infinispan.cache.web.jws-session-dist-async.war: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.preload() on object of type PersistenceManagerImpl
	at org.wildfly.clustering.service.FunctionalValueService.start(FunctionalValueService.java:70)
	at org.wildfly.clustering.service.AsynchronousServiceBuilder.lambda$start$0(AsynchronousServiceBuilder.java:99)
	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)
	at java.lang.Thread.run(Thread.java:748)
	at org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.preload() on object of type PersistenceManagerImpl
	at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:83)
	at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
	at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
	at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:165)
	at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
	at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:635)
	at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:624)
	at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:549)
	at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:232)
	at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:987)
	at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
	at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:653)
	at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:598)
	at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:462)
	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:448)
	at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:434)
	at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:60)
	at org.wildfly.clustering.infinispan.spi.service.CacheBuilder.lambda$build$0(CacheBuilder.java:83)
	at org.wildfly.clustering.service.SuppliedValueService.lambda$new$0(SuppliedValueService.java:42)
	at org.wildfly.clustering.service.FunctionalValueService.start(FunctionalValueService.java:68)
	... 7 more
Caused by: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for messageId=3 returned server error (status=0x85): java.lang.IllegalStateException: ISPN006016: Factory 'org.infinispan.server.hotrod.HotRodServer$ToEmptyBytesKeyValueFilterConverter' not found in server
	at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:363)
	at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:152)
	at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:138)
	at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:60)
	at org.infinispan.client.hotrod.impl.operations.IterationStartOperation.executeOperation(IterationStartOperation.java:72)
	at org.infinispan.client.hotrod.impl.operations.IterationStartOperation.executeOperation(IterationStartOperation.java:21)
	at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:56)
	at org.infinispan.client.hotrod.impl.iteration.RemoteCloseableIterator.startInternal(RemoteCloseableIterator.java:127)
	at org.infinispan.client.hotrod.impl.iteration.RemoteCloseableIterator.start(RemoteCloseableIterator.java:140)
	at org.infinispan.client.hotrod.impl.RemoteCacheImpl.retrieveEntries(RemoteCacheImpl.java:163)
	at org.infinispan.client.hotrod.impl.RemoteCacheImpl.retrieveEntries(RemoteCacheImpl.java:169)
	at org.infinispan.client.hotrod.impl.RemoteCacheImpl.retrieveEntries(RemoteCacheImpl.java:174)
	at org.infinispan.client.hotrod.impl.RemoteCacheImpl$KeySet.iterator(RemoteCacheImpl.java:562)
	at org.infinispan.persistence.remote.RemoteStore.process(RemoteStore.java:161)
	at org.infinispan.persistence.manager.PersistenceManagerImpl.preload(PersistenceManagerImpl.java:272)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
	... 26 more

      Link to the test console output: https://eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/jdg-func-session-externalization-eap/23/CONFIG=DOMAIN,container=eap,jdk=oracle-java-1.8,label_exp=RHEL7,strategy=COARSE/consoleFull

      This points to ISPN-8719 issue.

      After initial investigation, we concluded with help of vjuranek@redhat.com that the problem is the following: EAP 7.2.0.CD12.CR2 uses ISPN 9.1.7 JDG 7.1.2 uses ISPN 8.4.2 Meaning, EAP uses newer HotRod protocol version when communicating with JDG server.

      Update: The protocol versions turned out to be the same, but the ISPN-8719 issue is valid.

      Summary: The original issue, as presented on PM list was: EAP 7.2.0.CD12.CR2 cannot be configured to externalize sessions into JDG cluster. This has proven not to be correct. EAP 7.2.0.CD12.CR2 can be configured to externalize sessions into JDG cluster, the only non-working configuration is, when "preload" is set to "true". New Jira JBEAP-14607 has been opened for the "preload=true" issue.

      Attachments

        Issue Links

          is related to

          Bug - A problem that impairs or prevents the functions of the product. JBEAP-14607 Cache with passivation store=remote broken when using preload=true with combination of EAP 7.2.0.CD12.CR2 and JDG 7.1.2.GA

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Verified

          Bug - A problem that impairs or prevents the functions of the product. ISPN-8719 KeySet.(iterator|spliterator|stream) not compatible with versions before 9.1

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed

          Activity

            People

              rhn-engineering-rhusar Radoslav Husar
              mvinkler1@redhat.com Michal Vinkler
              Tommaso Borgato Tommaso Borgato
              Tommaso Borgato Tommaso Borgato
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: