Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-4717

Hot Rod 2.0 should add error codes for suspected nodes and stopping/stopped caches

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 7.0.0.CR1, 7.0.0.Final
    • 6.0.2.Final
    • None
    • None

    Description

      The way Hot Rod protocol deals with suspected exceptions is hacky. It inspects the error message to detect whether a SuspectException has been passed in. Instead, suspect exceptions should have a dedicated error code so that clients can handle appropriately.

      On top of that, another exception that should be handled more silently and failover is when a cache is stopping or is stopped. Currently, this produces the following log messages without affecting functionality Scrap that, it does get propagated to the client without being able to failover, so it's a bug:

      2014-09-11 08:11:04,984 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005003: Exception reported
java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
	at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
	at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
	at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
	at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
	at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
	at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
	at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
	at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
	at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
	at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
	at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
	at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
	at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
	at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
	at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
	at java.lang.Thread.run(Thread.java:744)
2014-09-11 08:11:04,990 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005009: Unexpected error before any request parameters read
io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
	at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:417)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
	at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
	at java.lang.Thread.run(Thread.java:744)
Caused by: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
	at org.infinispan.server.hotrod.HotRodDecoder.createServerException(HotRodDecoder.scala:204)
	at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:77)
	at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
	at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
	... 12 more
Caused by: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
	at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
	at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
	at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
	at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
	at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
	at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
	at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
	at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
	at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
	at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
	at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
	at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
	... 14 more
2014-09-11 08:11:04,991 WARN  [Codec20] (ForkThread-1,DistTopologyChangeUnderLoadTest) ISPN004005: Error received from the server: io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.

      Cache stopping/stopped should have a different error code too so that clients can handle it properly.

      Attachments

        Issue Links

          blocks

          Bug - A problem that impairs or prevents the functions of the product. ISPN-4707 On leave, Hot Rod client ends up with old cluster formation

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-3533 Design HotRod protocol version 2.0

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is related to

          Bug - A problem that impairs or prevents the functions of the product. ISPN-4707 On leave, Hot Rod client ends up with old cluster formation

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-1090 Hot Rod protocol improvements for version 2

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved
          relates to

          Bug - A problem that impairs or prevents the functions of the product. ISPN-4816 HotRod server should handle remote IllegalLifecycleExceptions

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed

          Activity

            People

              rh-ee-galder Galder Zamarreño
              rh-ee-galder Galder Zamarreño
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: