Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-10309

Convert Remaining Parts to Non Blocking & Reduce Thread Pools

    Details

    • Type: Enhancement
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 12.0.0.Final
    • Component/s: Core
    • Labels:
      None
    • Sprint:
      DataGrid Sprint #29, DataGrid Sprint #30, DataGrid Sprint #33, DataGrid Sprint #34, DataGrid Sprint #35, DataGrid Sprint #36, DataGrid Sprint #37, DataGrid Sprint #38, DataGrid Sprint #39

      Description

      We would love to get our thread pools down to a single CPU thread pool (size = numCores) and a blocking thread pool (arbitrarily large). We may also require a scheduler pool for various options as well (limited size 1-2?).

      To do this we need to remove remnants of our blocking code as possible. Possible issues for blocking are mostly around locks and io operations.

      The persistence layer was completed with ISPN-9722 so that is not an issue.

      The requirement around locking can be relaxed if the locks are guaranteed to be small in scope and do not wrap other blocking operations. An example would be a lock such as ones in CHM as long as we don't have large blocks for functional argument types.

      If code cannot be made non blocking we must offload the operation to the blocking thread pool. Care must be taken to ensure that once the blocking portion of code is completed that we switch back the to CPU thread pool as soon as possible. The listener API for example is violating this and will run code in Infinispan from any thread that completes the listener that could be done from a user.

        Gliffy Diagrams

          Attachments

            Issue Links

            1.
            State Transfer needs to be made non blocking Sub-task Closed Dan Berindei 10.1.0.Final Core, State Transfer
            2.
            Remove all the blocking Transport methods Sub-task Open Will Burns   Core
            3.
            We should ensure that when CompletionStages are completed they are continued only on Infinispan threads Sub-task Resolved Will Burns   Core
            4.
            Cache creation should be non blocking Sub-task New Will Burns   Core
            5.
            Evaluate locations that use CompletableFuture#join Sub-task Coding In Progress Will Burns    
            6.
            Non-blocking PendingLockManager Sub-task Closed Dan Berindei 10.0.0.CR2 Core, Transactions
            7.
            ExpirationManagerImpl Iteration removes from store synchronously Sub-task Closed Will Burns 10.0.0.Beta4  
            8.
            Consolidate Thread Pools Sub-task New Unassigned   Core
            9.
            Make sure code is not using random executors or fork join pool Sub-task Open Will Burns   Core
            10.
            Use stub for PersistenceManager when no stores are configured Sub-task Closed Will Burns 10.0.0.Beta4 Core
            11.
            Refactor ProtocolServer thread pools Sub-task New Unassigned   Server
            12.
            Add checkstyle that prevents use of *Async with no executor from CompletionStage Sub-task Closed Will Burns 10.0.0.Beta4, 10.0.0.Final  
            13.
            Scattered state transfer should be non blocking Sub-task New Unassigned    
            14.
            Convert usages of size to use sizeAsync Sub-task Closed Will Burns 10.1.0.Beta1  
            15.
            Add BlockHound to test suite to make sure cpu threads are never blocked Sub-task Closed Will Burns 11.0.0.Alpha2 Test Suite
            16.
            Add checkstyle to prevent invoking blocking methods from Flowable Sub-task Closed Will Burns 10.1.0.Beta1 Core
            17.
            Replace CompletionStageToPublisher with CompletionStageToSingle Sub-task Closed Will Burns 10.1.0.Beta1 Core
            18.
            Add interface to designate a thread is non blocking Sub-task Closed Will Burns 10.1.0.Beta1 Core
            19.
            Convert L1 to be non blocking Sub-task New Unassigned   L1
            20.
            GlobalConfigurationManager should use blocking threads when starting caches Sub-task Closed Will Burns 11.0.0.Alpha2 Core
            21.
            XSiteStateTransferProviderImpl is blocking when sending state Sub-task Closed Will Burns 11.0.0.Alpha2 Core, Cross-Site Replication
            22.
            Conflict Manager blocks while doing processing Sub-task New Unassigned    
            23.
            Convert async transport executor to the non blocking thread pool Sub-task Closed Will Burns 11.0.0.Alpha2  
            24.
            Add in (non)blocking thread pools Sub-task Closed Will Burns 11.0.0.Alpha2 Core
            25.
            InfinispanLock needs to block on a blocking thread but continue its operation on a non blocking thread Sub-task Closed Will Burns 11.0.0.Dev04 Core
            26.
            Convert Remote Command Executor to Non blocking/blocking thread executor Sub-task Closed Will Burns 11.0.0.Dev03 Core
            27.
            Requesting a cache that is starting will block Sub-task New Unassigned    
            28.
            BasicComponentRegistryImpl can block when starting a component Sub-task Closed Will Burns 11.0.0.Dev04 Core
            29.
            Counter creation should be non blocking Sub-task New Will Burns    
            30.
            Add blockhound support to hotrod server Sub-task Closed Will Burns 11.0.0.Dev03 Server
            31.
            PersistenceManagerImpl thread checks need to be updated Sub-task Closed Will Burns 11.0.0.Dev04  
            32.
            SegmentedBoundedOffHeapDataContainer#ensureSize should be marked as okay to be blocking Sub-task Closed Will Burns 11.0.0.Dev03  
            33.
            TransactionTable#calculateMinTopologyId should allow blocking Sub-task Closed Will Burns 11.0.0.Dev03  
            34.
            InvocationHelper should commit or rollback the transaction on a blocking thread Sub-task Closed Will Burns 11.0.0.Dev03  
            35.
            RecoveryManager API should be non blocking Sub-task Closed Will Burns 11.0.0.Dev03  
            36.
            Provide way for ClusterExecutor to use non blocking thread Sub-task Open Will Burns 12.0.0.Final Core
            37.
            Change all commands to not set canBlock Sub-task New Will Burns    
            38.
            TransactionCoordinator updated for non blocking Sub-task Closed Will Burns 11.0.0.Dev04  
            39.
            Convert detection of blocking or non blocking threads Sub-task Closed Will Burns 11.0.0.Dev04  
            40.
            PersistenceManagerImpl locks should be able to block Sub-task Closed Unassigned 11.0.0.Dev05  
            41.
            HotRod Server transaction is blocking Sub-task Closed Will Burns 11.0.0.CR1  
            42.
            Replace HotRod worker thread pool Sub-task Closed Will Burns 11.0.0.CR1  
            43.
            Convert ClientListenerRegistry to use non blocking calls Sub-task Closed Will Burns 11.0.0.Dev04  
            44.
            Add BlockingHandler to simplify running blocking operations Sub-task Closed Will Burns 11.0.0.Dev05  
            45.
            Add checkstyle to prevent invoking parallelStream Sub-task New Will Burns    
            46.
            Cluster Locks should be non blocking Sub-task Resolved Unassigned   Clustered Locks
            47.
            Remove extra thread in BlockingTaskAwareExecutorServiceImpl Sub-task Closed Will Burns 11.0.0.Dev04 Core
            48.
            Fix blocking LimitedExecutor#addPermit test failures Sub-task Closed Will Burns 11.0.0.Dev05 Core

              Activity

                People

                • Assignee:
                  william.burns Will Burns
                  Reporter:
                  william.burns Will Burns
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  5 Start watching this issue

                  Dates

                  • Created:
                    Updated: