Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-12571

jcache/tck-runner-remote random failures starting server

XMLWordPrintable

      The jcache/tck-runner-remote module starts an Infinispan server with the default configuration. Sometimes, the server fails to start:

      2020-12-10 08:33:49,367 INFO  (main) [org.infinispan.CLUSTER] ISPN000079: Channel cluster local address is rhos-infinispan-ci-4-5381, physical addresses are [10.0.148.134:7800]
      2020-12-10 08:33:49,416 INFO  (main) [org.infinispan.CONTAINER] ISPN000390: Persisted state, version=12.0.0-SNAPSHOT timestamp=2020-12-10T13:33:49.414476Z
      2020-12-10 08:33:49,737 INFO  (main) [org.jboss.threads] JBoss Threads version 2.3.3.Final
      2020-12-10 08:33:49,811 INFO  (main) [org.infinispan.CONTAINER] ISPN000104: Using EmbeddedTransactionManager
      2020-12-10 08:33:50,362 WARN  (blocking-thread--p3-t1) [org.infinispan.encoding.impl.StorageConfigurationManager] ISPN000599: Configuration for cache 'ensure_mbeanserver_created_cache' does not define the encoding for keys or values. If you use operations that require data conversion or queries, you should configure the cache with a specific MediaType for keys or values.
      2020-12-10 08:33:50,369 WARN  (blocking-thread--p3-t2) [org.infinispan.encoding.impl.StorageConfigurationManager] ISPN000599: Configuration for cache 'SampleCache' does not define the encoding for keys or values. If you use operations that require data conversion or queries, you should configure the cache with a specific MediaType for keys or values.
      2020-12-10 08:33:50,405 INFO  (main) [org.infinispan.CLUSTER] ISPN000080: Disconnecting JGroups channel cluster
      2020-12-10 08:33:50,417 INFO  (main) [org.infinispan.CONTAINER] ISPN000390: Persisted state, version=12.0.0-SNAPSHOT timestamp=2020-12-10T13:33:50.416703Z
      2020-12-10 08:33:50,422 FATAL (main) [org.infinispan.SERVER] ISPN080028: Infinispan Server failed to start java.util.concurrent.ExecutionException: org.infinispan.manager.EmbeddedCacheManagerStartupException: org.infinispan.commons.CacheConfigurationException: Error starting component org.infinispan.globalstate.GlobalConfigurationManager
      	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
      	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
      	at org.infinispan.server.Bootstrap.runInternal(Bootstrap.java:158)
      	at org.infinispan.server.tool.Main.run(Main.java:98)
      	at org.infinispan.server.Bootstrap.main(Bootstrap.java:46)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at org.infinispan.server.loader.Loader.run(Loader.java:103)
      	at org.infinispan.server.loader.Loader.main(Loader.java:48)
      Caused by: org.infinispan.manager.EmbeddedCacheManagerStartupException: org.infinispan.commons.CacheConfigurationException: Error starting component org.infinispan.globalstate.GlobalConfigurationManager
      	at org.infinispan.manager.DefaultCacheManager.internalStart(DefaultCacheManager.java:751)
      	at org.infinispan.manager.DefaultCacheManager.start(DefaultCacheManager.java:717)
      	at org.infinispan.server.SecurityActions.lambda$startCacheManager$1(SecurityActions.java:67)
      	at org.infinispan.security.Security.doPrivileged(Security.java:45)
      	at org.infinispan.server.SecurityActions.doPrivileged(SecurityActions.java:39)
      	at org.infinispan.server.SecurityActions.startCacheManager(SecurityActions.java:70)
      	at org.infinispan.server.Server.run(Server.java:346)
      	... 9 more
      Caused by: org.infinispan.commons.CacheConfigurationException: Error starting component org.infinispan.globalstate.GlobalConfigurationManager
      	at org.infinispan.factories.impl.BasicComponentRegistryImpl.startWrapper(BasicComponentRegistryImpl.java:572)
      	at org.infinispan.factories.impl.BasicComponentRegistryImpl.access$700(BasicComponentRegistryImpl.java:30)
      	at org.infinispan.factories.impl.BasicComponentRegistryImpl$ComponentWrapper.running(BasicComponentRegistryImpl.java:787)
      	at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:341)
      	at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:237)
      	at org.infinispan.manager.DefaultCacheManager.internalStart(DefaultCacheManager.java:746)
      	... 15 more
      Caused by: java.util.concurrent.CompletionException: org.infinispan.commons.CacheConfigurationException: ISPN000502: Error while persisting global configuration state
      	at org.infinispan.util.concurrent.CompletionStages.join(CompletionStages.java:81)
      	at org.infinispan.globalstate.impl.GlobalConfigurationManagerImpl.lambda$start$0(GlobalConfigurationManagerImpl.java:115)
      	at org.infinispan.cache.impl.EncoderCache.lambda$forEach$7(EncoderCache.java:762)
      	at java.base/java.util.concurrent.ConcurrentMap.forEach(ConcurrentMap.java:122)
      	at org.infinispan.cache.impl.AbstractDelegatingCache.forEach(AbstractDelegatingCache.java:479)
      	at org.infinispan.cache.impl.AbstractDelegatingCache.forEach(AbstractDelegatingCache.java:479)
      	at org.infinispan.cache.impl.AbstractDelegatingCache.forEach(AbstractDelegatingCache.java:479)
      	at org.infinispan.cache.impl.EncoderCache.forEach(EncoderCache.java:759)
      	at org.infinispan.globalstate.impl.GlobalConfigurationManagerImpl.start(GlobalConfigurationManagerImpl.java:106)
      	at org.infinispan.globalstate.impl.CorePackageImpl$2.start(CorePackageImpl.java:59)
      	at org.infinispan.globalstate.impl.CorePackageImpl$2.start(CorePackageImpl.java:48)
      	at org.infinispan.factories.impl.BasicComponentRegistryImpl.invokeStart(BasicComponentRegistryImpl.java:604)
      	at org.infinispan.factories.impl.BasicComponentRegistryImpl.doStartWrapper(BasicComponentRegistryImpl.java:595)
      	at org.infinispan.factories.impl.BasicComponentRegistryImpl.startWrapper(BasicComponentRegistryImpl.java:564)
      	... 20 more
      Caused by: org.infinispan.commons.CacheConfigurationException: ISPN000502: Error while persisting global configuration state
      	at org.infinispan.globalstate.impl.OverlayLocalConfigurationStorage.persistConfigurations(OverlayLocalConfigurationStorage.java:156)
      	at org.infinispan.globalstate.impl.OverlayLocalConfigurationStorage.storeCaches(OverlayLocalConfigurationStorage.java:130)
      	at org.infinispan.globalstate.impl.OverlayLocalConfigurationStorage.lambda$createCache$2(OverlayLocalConfigurationStorage.java:79)
      	at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:642)
      	at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478)
      	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
      	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
      	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
      	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
      	at java.base/java.lang.Thread.run(Thread.java:834)
      Caused by: org.infinispan.commons.CacheConfigurationException: ISPN000508: Cannot rename file /home/infinispan/workspace/Infinispan_PR-8921/jcache/tck-runner-remote/target/infinispan-server/server/data/caches9096240479751965790.tmp to /home/infinispan/workspace/Infinispan_PR-8921/jcache/tck-runner-remote/target/infinispan-server/server/data/caches.xml
      	at org.infinispan.globalstate.impl.OverlayLocalConfigurationStorage.persistConfigurations(OverlayLocalConfigurationStorage.java:153)
      	... 9 more
      Caused by: java.nio.channels.OverlappingFileLockException
      	at java.base/sun.nio.ch.FileLockTable.checkList(FileLockTable.java:229)
      	at java.base/sun.nio.ch.FileLockTable.add(FileLockTable.java:123)
      	at java.base/sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1109)
      	at java.base/java.nio.channels.FileChannel.lock(FileChannel.java:1063)
      	at org.infinispan.commons.util.Util.renameTempFile(Util.java:1087)
      	at org.infinispan.globalstate.impl.OverlayLocalConfigurationStorage.persistConfigurations(OverlayLocalConfigurationStorage.java:151)
      	... 9 more
      

      Probably unrelated, but still something that to fix: because of the default configuration, the servers started by 2 builds running in the same network will form a cluster:

      2020-12-10 08:33:49,359 INFO  (main) [org.infinispan.CLUSTER] ISPN000094: Received new cluster view for channel cluster: [rhos-infinispan-ci-3-54013|1] (2) [rhos-infinispan-ci-3-54013, rhos-infinispan-ci-4-5381]
      

            Unassigned Unassigned
            dberinde@redhat.com Dan Berindei (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: