Uploaded image for project: 'Red Hat Fuse'
  1. Red Hat Fuse
  2. ENTESB-3907

Deployment takes a lot of time in latest patched Fabric environment with ~70 profile.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • jboss-fuse-6.2
    • jboss-fuse-6.1
    • Fabric8 v1
    • None
    • % %

      • Customer has ~70 profiles, updating a profile and thus applying it to fabric takes around 10 to 15 minutes for deployment. CPU usage also spikes to 90 to 100 % during deployment.
        "pool-367-thread-1" prio=6 tid=0x000000000c2d9800 nid=0x3198 runnable [0x0000000014f2d000]
           java.lang.Thread.State: RUNNABLE
        	at org.eclipse.jgit.internal.storage.file.PackFile.idx(PackFile.java:164)
        	- locked <0x00000000c3747670> (a org.eclipse.jgit.internal.storage.file.PackFile)
        	at org.eclipse.jgit.internal.storage.file.PackFile.get(PackFile.java:255)
        	at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openObject1(ObjectDirectory.java:371)
        	at org.eclipse.jgit.internal.storage.file.FileObjectDatabase.openObjectImpl1(FileObjectDatabase.java:173)
        	at org.eclipse.jgit.internal.storage.file.FileObjectDatabase.openObject(FileObjectDatabase.java:158)
        	at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:145)
        	at org.eclipse.jgit.revwalk.RevWalk.getCachedBytes(RevWalk.java:853)
        	at org.eclipse.jgit.revwalk.RevCommit.parseHeaders(RevCommit.java:143)
        	at org.eclipse.jgit.revwalk.RewriteTreeFilter.include(RewriteTreeFilter.java:109)
        	at org.eclipse.jgit.revwalk.PendingGenerator.next(PendingGenerator.java:141)
        	at org.eclipse.jgit.revwalk.BlockRevQueue.<init>(BlockRevQueue.java:65)
        	at org.eclipse.jgit.revwalk.FIFORevQueue.<init>(FIFORevQueue.java:64)
        	at org.eclipse.jgit.revwalk.StartGenerator.next(StartGenerator.java:157)
        	at org.eclipse.jgit.revwalk.RevWalk.next(RevWalk.java:414)
        	at org.gitective.core.CommitUtils.getLastCommit(CommitUtils.java:418)
        	at io.fabric8.git.internal.CachingGitDataStore.addProfileData(CachingGitDataStore.java:162)
        	at io.fabric8.git.internal.CachingGitDataStore.addProfileData(CachingGitDataStore.java:152)
        	at io.fabric8.git.internal.CachingGitDataStore.addProfileData(CachingGitDataStore.java:152)
        	at io.fabric8.git.internal.CachingGitDataStore.pouplateVersionData(CachingGitDataStore.java:132)
        	at io.fabric8.git.internal.CachingGitDataStore$1$1.call(CachingGitDataStore.java:91)
        	at io.fabric8.git.internal.CachingGitDataStore$1$1.call(CachingGitDataStore.java:88)
        	at io.fabric8.git.internal.GitDataStore.gitOperation(GitDataStore.java:934)
        	- locked <0x00000000c3469200> (a java.lang.Object)
        	at io.fabric8.git.internal.GitDataStore.gitOperation(GitDataStore.java:901)
        	at io.fabric8.git.internal.GitDataStore.gitOperation(GitDataStore.java:896)
        	at io.fabric8.git.internal.CachingGitDataStore$1.load(CachingGitDataStore.java:88)
        	at io.fabric8.git.internal.CachingGitDataStore$1.load(CachingGitDataStore.java:85)
        	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3522)
        	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2315)
        	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2278)
        	- locked <0x00000000c346f9e0> (a com.google.common.cache.LocalCache$StrongEntry)
        	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2193)
        	at com.google.common.cache.LocalCache.get(LocalCache.java:3932)
        	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3936)
        	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4806)
        	at io.fabric8.git.internal.CachingGitDataStore.getVersionData(CachingGitDataStore.java:116)
        	at io.fabric8.git.internal.CachingGitDataStore.hasProfile(CachingGitDataStore.java:187)
        	at io.fabric8.internal.VersionImpl.getProfile(VersionImpl.java:100)
        	at io.fabric8.internal.ContainerImpl$ContainerProfile.getParents(ContainerImpl.java:271)
        	at io.fabric8.internal.ProfileOverlayImpl.fillParentProfiles(ProfileOverlayImpl.java:248)
        	at io.fabric8.internal.ProfileOverlayImpl.getInheritedProfiles(ProfileOverlayImpl.java:243)
        	at io.fabric8.internal.ProfileOverlayImpl.getConfigurations(ProfileOverlayImpl.java:346)
        	at io.fabric8.internal.ProfileOverlayImpl.getContainerConfiguration(ProfileOverlayImpl.java:121)
        	at io.fabric8.internal.ProfileImpl.getContainerConfigList(ProfileImpl.java:149)
        	at io.fabric8.internal.ProfileOverlayImpl.getRepositories(ProfileOverlayImpl.java:106)
        	at io.fabric8.features.FabricFeaturesServiceImpl.listInstalledRepositories(FabricFeaturesServiceImpl.java:327)
        	at io.fabric8.features.FabricFeaturesServiceImpl.run(FabricFeaturesServiceImpl.java:93)
        	- locked <0x00000000c3202c18> (a io.fabric8.features.FabricFeaturesServiceImpl)
        	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        	at java.util.concurrent.FutureTask.run(Unknown Source)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        	at java.lang.Thread.run(Unknown Source)
        
           Locked ownable synchronizers:
        	- <0x00000000c3202e80> (a java.util.concurrent.ThreadPoolExecutor$Worker)
        
        
        "pool-365-thread-1" prio=6 tid=0x000000000c2ed000 nid=0x2960 waiting for monitor entry [0x00000000123cf000]
           java.lang.Thread.State: BLOCKED (on object monitor)
        	at io.fabric8.git.internal.GitDataStore.gitOperation(GitDataStore.java:906)
        	- waiting to lock <0x00000000c3469200> (a java.lang.Object)
        	at io.fabric8.git.internal.GitDataStore.gitOperation(GitDataStore.java:901)
        	at io.fabric8.git.internal.GitDataStore.gitOperation(GitDataStore.java:891)
        	at io.fabric8.git.internal.GitDataStore.pull(GitDataStore.java:1251)
        	at io.fabric8.git.internal.GitDataStore$1.run(GitDataStore.java:186)
        	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        	at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
        	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
        	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        	at java.lang.Thread.run(Unknown Source)
        
           Locked ownable synchronizers:
        	- <0x00000000c3468a90> (a java.util.concurrent.ThreadPoolExecutor$Worker)
        
      • As described in https://issues.jboss.org/browse/ENTESB-2581, this issue should be fixed with R2P1 and a property "gitGcOnLoad=true" within root container as described in ENTESB-2581.
      • But User is with latest Patch R2P6 and still see the delay in deployment with above thread dump.
      • User perform deployment in child-container.
      • System Details:
        Processor: Intel Xeon X5675 3.75 Ghz(2 Processors)
        RAM: 16 GB
        System Type: 64 bit OS
        OS: Windows Server 2008 R2 Enterprise SP1
        

              janstey@redhat.com Jonathan Anstey
              rhn-support-cpandey Chandra Shekhar Pandey (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: