Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-11172

GracefulShutdownRestartIT fails

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 10.1.3.Final, 11.0.0.Alpha2
    • 10.1.0.Final
    • None
    • None

    Description

      Error Message

Cluster did not shutdown within timeout

Stacktrace

java.lang.AssertionError: Cluster did not shutdown within timeout
	at org.infinispan.commons.util.Eventually.lambda$eventually$0(Eventually.java:33)
	at org.infinispan.commons.util.Eventually.eventually(Eventually.java:25)
	at org.infinispan.commons.util.Eventually.eventually(Eventually.java:33)
	at org.infinispan.server.resilience.GracefulShutdownRestartIT.testGracefulShutdownRestart(GracefulShutdownRestartIT.java:50)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.infinispan.server.test.InfinispanServerTestMethodRule$1.evaluate(InfinispanServerTestMethodRule.java:69)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.infinispan.server.test.InfinispanServerRule$1.evaluate(InfinispanServerRule.java:90)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
	at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)

Standard Output

[OK:   131, KO:     0, SKIP:     0] Test starting: GracefulShutdownRestartIT.testGracefulShutdownRestart
[0] STDOUT: 12:36:05,513 WARN  (async-thread--p2-t6) [CONFIG] ISPN000564: Configured store 'SingleFileStore' is segmented and may use a large number of file descriptors&amp#27;[m
[0] STDOUT: 12:36:05,514 WARN  (async-thread--p2-t6) [CONFIG] ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup&amp#27;[m
[0] STDOUT: 12:36:05,515 WARN  (async-thread--p2-t6) [CONFIG] ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup&amp#27;[m
[1] STDOUT: 12:36:05,558 WARN  (remote-thread--p3-t2) [CONFIG] ISPN000564: Configured store 'SingleFileStore' is segmented and may use a large number of file descriptors&amp#27;[m
[1] STDOUT: 12:36:05,560 WARN  (remote-thread--p3-t2) [CONFIG] ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup&amp#27;[m
[1] STDOUT: 12:36:05,562 WARN  (remote-thread--p3-t2) [CONFIG] ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup&amp#27;[m
[0] STDOUT: 12:36:05,771 WARN  (jgroups-13,06978f4151c0-63390) [CONFIG] ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup&amp#27;[m
[0] STDOUT: 12:36:05,776 WARN  (jgroups-13,06978f4151c0-63390) [CONFIG] ISPN000149: Fetch persistent state and purge on startup are both disabled, cache may contain stale entries on startup&amp#27;[m
[0] STDOUT: 12:36:05,883 INFO  (transport-thread--p5-t7) [CLUSTER] [Context=C3F31F9D169B25D0C68FE5A26A6600FFA0E2E3F4]ISPN100002: Starting rebalance with members [63781b79183e-43839, 06978f4151c0-63390], phase READ_OLD_WRITE_ALL, topology id 2&amp#27;[m
[0] STDOUT: 12:36:05,963 INFO  (remote-thread--p3-t1) [CLUSTER] [Context=C3F31F9D169B25D0C68FE5A26A6600FFA0E2E3F4]ISPN100009: Advancing to rebalance phase READ_ALL_WRITE_ALL, topology id 3&amp#27;[m
[0] STDOUT: 12:36:05,978 INFO  (remote-thread--p3-t1) [CLUSTER] [Context=C3F31F9D169B25D0C68FE5A26A6600FFA0E2E3F4]ISPN100009: Advancing to rebalance phase READ_NEW_WRITE_ALL, topology id 4&amp#27;[m
[0] STDOUT: 12:36:05,985 INFO  (async-thread--p2-t17) [CLUSTER] [Context=C3F31F9D169B25D0C68FE5A26A6600FFA0E2E3F4]ISPN100010: Finished rebalance with members [63781b79183e-43839, 06978f4151c0-63390], topology id 5&amp#27;[m
[0] STDERR: WARNING: An illegal reflective access operation has occurred
[0] STDERR: WARNING: Illegal reflective access by protostream.com.google.protobuf.UnsafeUtil (file:/opt/infinispan/lib/protostream-4.3.1.Final.jar) to field java.nio.Buffer.address
[0] STDERR: WARNING: Please consider reporting this to the maintainers of protostream.com.google.protobuf.UnsafeUtil
[0] STDERR: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
[0] STDERR: WARNING: All illegal access operations will be denied in a future release
[1] STDERR: WARNING: An illegal reflective access operation has occurred
[1] STDERR: WARNING: Illegal reflective access by protostream.com.google.protobuf.UnsafeUtil (file:/opt/infinispan/lib/protostream-4.3.1.Final.jar) to field java.nio.Buffer.address
[1] STDERR: WARNING: Please consider reporting this to the maintainers of protostream.com.google.protobuf.UnsafeUtil
[1] STDERR: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
[1] STDERR: WARNING: All illegal access operations will be denied in a future release
[0] STDOUT: 12:36:06,811 INFO  (SINGLE_PORT-ServerIO-8-2) [CLUSTER] [Context=___protobuf_metadata]ISPN100008: Updating cache members list [63781b79183e-43839], topology id 6&amp#27;[m
[0] STDOUT: 12:36:06,860 INFO  (SINGLE_PORT-ServerIO-8-2) [CLUSTER] [Context=C3F31F9D169B25D0C68FE5A26A6600FFA0E2E3F4]ISPN100008: Updating cache members list [63781b79183e-43839], topology id 6&amp#27;[m
[0] STDOUT: 12:36:06,898 INFO  (SINGLE_PORT-ServerIO-8-2) [CLUSTER] [Context=memcachedCache]ISPN100008: Updating cache members list [63781b79183e-43839], topology id 6&amp#27;[m
[1] STDOUT: 12:36:06,904 WARN  (remote-thread--p3-t2) [CLUSTER] ISPN000071: Caught exception when handling command CacheTopologyControlCommand{cache=___script_cache, type=SHUTDOWN_PERFORM, sender=null, joinInfo=null, topologyId=0, rebalanceId=0, currentCH=null, pendingCH=null, availabilityMode=null, phase=null, actualMembers=null, throwable=null, viewId=0} java.lang.NullPointerException
[1] STDOUT: 	at org.infinispan.topology.LocalTopologyManagerImpl.handleCacheShutdown(LocalTopologyManagerImpl.java:743)
[1] STDOUT: 	at org.infinispan.topology.CacheTopologyControlCommand.doPerform(CacheTopologyControlCommand.java:189)
[1] STDOUT: 	at org.infinispan.topology.CacheTopologyControlCommand.invokeAsync(CacheTopologyControlCommand.java:160)
[1] STDOUT: 	at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler$ReplicableCommandRunner.run(GlobalInboundInvocationHandler.java:160)
[1] STDOUT: 	at org.infinispan.util.concurrent.BlockingTaskAwareExecutorServiceImpl$RunnableWrapper.run(BlockingTaskAwareExecutorServiceImpl.java:215)
[1] STDOUT: 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[1] STDOUT: 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[1] STDOUT: 	at java.base/java.lang.Thread.run(Thread.java:834)
[1] STDOUT: &amp#27;[m

      Attachments

        Issue Links

          is related to

          Bug - A problem that impairs or prevents the functions of the product. ISPN-6325 Other node shutting down caused local exception

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              ttarrant@redhat.com Tristan Tarrant
              ttarrant@redhat.com Tristan Tarrant
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: