-
Bug
-
Resolution: Done
-
Major
-
12.0.1.Final
-
None
I'm running 3 Infinispan nodes with distributed caches (2 cache owners). After enabling security with roles/permissions I get a lot of stack traces when an Infinispan node is starting up. The errors can only be observed in the beginning then caches are rebalancing. Once the startup is completed no more errors are shown and everything seems to be working as expected.
Feb 18 17:10:52 ispn-3 bash[261064]: 17:10:52,364 WARN (jgroups-26,ispn-3) [org.infinispan.CLUSTER] ISPN000071: Caught exception when handling command org.infinispan.manager.impl.ReplicableManagerFunctionCommand@3f7ffaaa java.lang.SecurityException: ISPN000287: Unauthorized access: subject 'null' lacks 'LIFECYCLE' permission Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.security.impl.AuthorizationHelper.checkPermission(AuthorizationHelper.java:98) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:660) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:643) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:532) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:510) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.manager.impl.AbstractDelegatingEmbeddedCacheManager.getCache(AbstractDelegatingEmbeddedCacheManager.java:193) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.manager.impl.UnwrappingEmbeddedCacheManager.getCache(UnwrappingEmbeddedCacheManager.java:25) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.notifications.cachelistener.cluster.ClusterListenerReplicateCallable.apply(ClusterListenerReplicateCallable.java:75) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.notifications.cachelistener.cluster.ClusterListenerReplicateCallable.apply(ClusterListenerReplicateCallable.java:39) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.manager.impl.ReplicableManagerFunctionCommand.invokeAsync(ReplicableManagerFunctionCommand.java:39) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$ReplicableCommandRunner.run(GlobalInboundInvocationHandler.java:255) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleReplicableCommand(GlobalInboundInvocationHandler.java:174) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleFromCluster(GlobalInboundInvocationHandler.java:113) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processRequest(JGroupsTransport.java:1383) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1307) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$300(JGroupsTransport.java:131) Feb 18 17:10:52 ispn-3 bash[261064]: at org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.up(JGroupsTransport.java:1450) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.JChannel.up(JChannel.java:784) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:913) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.FRAG3.up(FRAG3.java:165) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.FlowControl.up(FlowControl.java:351) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.FlowControl.up(FlowControl.java:359) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.pbcast.GMS.up(GMS.java:876) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:243) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1049) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.UNICAST3.addMessage(UNICAST3.java:772) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:753) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:405) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:592) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:132) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.FailureDetection.up(FailureDetection.java:186) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:254) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.MERGE3.up(MERGE3.java:281) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.Discovery.up(Discovery.java:300) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.protocols.TP.passMessageUp(TP.java:1396) Feb 18 17:10:52 ispn-3 bash[261064]: at org.jgroups.util.SubmitToThreadPool$SingleMessageHandler.run(SubmitToThreadPool.java:87) Feb 18 17:10:52 ispn-3 bash[261064]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) Feb 18 17:10:52 ispn-3 bash[261064]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) Feb 18 17:10:52 ispn-3 bash[261064]: at java.base/java.lang.Thread.run(Thread.java:834) Feb 18 17:10:52 ispn-3 bash[261064]:
Similar errors occur when the /metrics endpoint is called during startup. Again, no issues once startup is completed:
Feb 18 16:28:49 ispn-2 bash[255577]: 16:28:49,126 ERROR (SINGLE_PORT-ServerIO-4-1) [org.infinispan.stats.impl.ClusterCacheStatsImpl] Could not execute cluster wide cache stats operation java.util.concurrent.CompletionExce[685/1918] .infinispan.commons.CacheException: java.lang.SecurityException: ISPN000287: Unauthorized access: subject 'null' lacks 'LIFECYCLE' permission Feb 18 16:28:49 ispn-2 bash[255577]: at java.base/java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:412) Feb 18 16:28:49 ispn-2 bash[255577]: at java.base/java.util.concurrent.CompletableFuture.join(CompletableFuture.java:2044) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.stats.impl.ClusterCacheStatsImpl.updateStats(ClusterCacheStatsImpl.java:118) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.stats.impl.AbstractClusterStats.fetchClusterWideStatsIfNeeded(AbstractClusterStats.java:117) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.stats.impl.AbstractClusterStats.getStat(AbstractClusterStats.java:210) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.stats.impl.AbstractClusterStats.getStatAsLong(AbstractClusterStats.java:201) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.stats.impl.ClusterCacheStatsImpl.getAverageReadTime(ClusterCacheStatsImpl.java:155) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.factories.impl.MBeanMetadata$AttributeMetadata.lambda$getter$0(MBeanMetadata.java:137) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.metrics.impl.MetricsCollector.lambda$registerMetrics$0(MetricsCollector.java:139) Feb 18 16:28:49 ispn-2 bash[255577]: at io.smallrye.metrics.exporters.OpenMetricsExporter.createSimpleValueLine(OpenMetricsExporter.java:492) Feb 18 16:28:49 ispn-2 bash[255577]: at io.smallrye.metrics.exporters.OpenMetricsExporter.exposeEntries(OpenMetricsExporter.java:192) Feb 18 16:28:49 ispn-2 bash[255577]: at io.smallrye.metrics.exporters.OpenMetricsExporter.getEntriesForScope(OpenMetricsExporter.java:158) Feb 18 16:28:49 ispn-2 bash[255577]: at io.smallrye.metrics.exporters.OpenMetricsExporter.exportAllScopes(OpenMetricsExporter.java:109) Feb 18 16:28:49 ispn-2 bash[255577]: at io.smallrye.metrics.MetricsRequestHandler.handleRequest(MetricsRequestHandler.java:116) Feb 18 16:28:49 ispn-2 bash[255577]: at io.smallrye.metrics.MetricsRequestHandler.handleRequest(MetricsRequestHandler.java:73) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.rest.resources.MetricsResource.metrics(MetricsResource.java:92) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.rest.framework.impl.RestDispatcherImpl.dispatch(RestDispatcherImpl.java:75) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.rest.RestRequestHandler.handleRestRequest(RestRequestHandler.java:139) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.rest.RestRequestHandler.channelRead0(RestRequestHandler.java:90) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.rest.RestRequestHandler.channelRead0(RestRequestHandler.java:37) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.handler.codec.http.HttpServerKeepAliveHandler.channelRead(HttpServerKeepAliveHandler.java:64) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.rest.CorsHandler.channelRead(CorsHandler.java:99) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.rest.StreamCorrelatorHandler.channelRead(StreamCorrelatorHandler.java:27) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) Feb 18 16:28:49 ispn-2 bash[255577]: at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) Feb 18 16:28:49 ispn-2 bash[255577]: at java.base/java.lang.Thread.run(Thread.java:834) Feb 18 16:28:49 ispn-2 bash[255577]: Caused by: org.infinispan.commons.CacheException: java.lang.SecurityException: ISPN000287: Unauthorized access: subject 'null' lacks 'LIFECYCLE' permission Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.stats.impl.ClusterCacheStatsImpl.lambda$updateStats$0(ClusterCacheStatsImpl.java:109) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.manager.impl.AllClusterExecutor.lambda$submitConsumer$6(AllClusterExecutor.java:192) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.manager.impl.AbstractClusterExecutor.consumeResponse(AbstractClusterExecutor.java:65) Feb 18 16:28:49 ispn-2 bash[255577]: at org.infinispan.manager.impl.AllClusterExecutor.lambda$submitConsumer$7(AllClusterExecutor.java:191)
