Uploaded image for project: 'Application Server 7'
  1. Application Server 7
  2. AS7-5393

"Problems unmarshalling remote command from byte buffer: java.io.StreamCorruptedException: Unexpected byte found when reading an object: 50"

    XMLWordPrintable

Details

    Description

      As noticed by Kabir in AS7 master TS runs upon failover:

      http://lightning.mw.lab.eng.bos.redhat.com/jenkins/job/as7-master-testsuite-ip6/3337/testReport/junit/org.jboss.as.test.clustering.cluster.web/ReplicationWebFailoverTestCase%28UDP%29/testGracefulUndeployFailover/

      23:00:59,836 WARN  [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (OOB-20,null) Problems unmarshalling remote command from byte buffer: java.io.StreamCorruptedException: Unexpected byte found when reading an object: 50
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:750) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.ModularClassResolver.resolveClass(ModularClassResolver.java:99) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:935) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1205) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
	at org.infinispan.marshall.exts.ReplicableCommandExternalizer.readParameters(ReplicableCommandExternalizer.java:119)
	at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:162)
	at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:66)
	at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:395)
	at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:224)
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
	at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:163)
	at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:114)
	at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:104)
	at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50)
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:200)
	at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:465)
	at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:372)
	at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247)
	at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:601)
	at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130)
	at org.jgroups.JChannel.up(JChannel.java:715)
	at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020)
	at org.jgroups.protocols.RSVP.up(RSVP.java:179)
	at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
	at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
	at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)
	at org.jgroups.protocols.pbcast.GMS.up(GMS.java:899)
	at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244)
	at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:744)
	at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:414)
	at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:572)
	at org.jgroups.protocols.BARRIER.up(BARRIER.java:102)
	at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
	at org.jgroups.protocols.FD.up(FD.java:273)
	at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288)
	at org.jgroups.protocols.MERGE2.up(MERGE2.java:205)
	at org.jgroups.protocols.Discovery.up(Discovery.java:359)
	at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2568)
	at org.jgroups.protocols.TP.passMessageUp(TP.java:1211)
	at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1775)
	at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1748)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_29]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_29]
	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_29]

[0m[31m23:00:59,943 ERROR [org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-3,node-0/web) ISPN000172: Failed to prepare view CacheView{viewId=4, members=[node-1/web, node-0/web]} for cache  default-host/distributable, rolling back to view CacheView{viewId=3, members=[node-1/web]}: java.util.concurrent.ExecutionException: org.infinispan.CacheException: Problems invoking command.
	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) [rt.jar:1.6.0_29]
	at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_29]
	at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterPrepareView(CacheViewsManagerImpl.java:319) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterInstallView(CacheViewsManagerImpl.java:250) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at org.infinispan.cacheviews.CacheViewsManagerImpl$ViewInstallationTask.call(CacheViewsManagerImpl.java:876) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [rt.jar:1.6.0_29]
	at java.util.concurrent.FutureTask.run(FutureTask.java:138) [rt.jar:1.6.0_29]
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_29]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_29]
	at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_29]
Caused by: org.infinispan.CacheException: Problems invoking command.
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:215) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:465) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:372) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:601) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.JChannel.up(JChannel.java:715) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.RSVP.up(RSVP.java:179) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.FRAG2.up(FRAG2.java:181) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.FlowControl.up(FlowControl.java:418) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.FlowControl.up(FlowControl.java:400) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.pbcast.GMS.up(GMS.java:899) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:744) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:414) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:572) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.BARRIER.up(BARRIER.java:102) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.FD.up(FD.java:273) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.MERGE2.up(MERGE2.java:205) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.Discovery.up(Discovery.java:359) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2568) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.TP.passMessageUp(TP.java:1211) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1775) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1748) [jgroups-3.1.0.Final.jar:3.1.0.Final]
	... 3 more
Caused by: java.io.StreamCorruptedException: Unexpected byte found when reading an object: 50
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:750) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.ModularClassResolver.resolveClass(ModularClassResolver.java:99) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:935) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1205) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
	at org.infinispan.marshall.exts.ReplicableCommandExternalizer.readParameters(ReplicableCommandExternalizer.java:119) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:162) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:66) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:395) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:224) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.15.GA.jar:1.3.15.GA]
	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
	at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:163) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:114) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:104) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:200) [infinispan-core-5.1.6.FINAL.jar:5.1.6.FINAL]
	... 29 more

      Attachments

        Issue Links

          incorporates

          Bug - A problem that impairs or prevents the functions of the product. ISPN-2330 JBossMarshaller uses wrong class resolver after stop/start

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. ISPN-2297 Cache restart doesn't work properly

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. ISPN-2418 CLONE - Cache restart doesn't work properly

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is related to

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. ISPN-2208 Upgrade to latest JBoss Marshalling 1.3.x

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          relates to

          Bug - A problem that impairs or prevents the functions of the product. JBMAR-143 missing read of next command for ID_CLEAR_INSTANCE_CACHE case in RiverUnmarshaller.doReadObject()

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-2214 Dump marshalled and unmarshalled payloads for tracing

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Activity

            People

              smarlow1@redhat.com Scott Marlow
              rhn-engineering-rhusar Radoslav Husar
              Votes:
              2 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: