Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-10309

Convert Remaining Parts to Non Blocking & Reduce Thread Pools

    XMLWordPrintable

Details

    • Enhancement
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • 12.1.8.Final
    • Core
    • None
    • DataGrid Sprint #29, DataGrid Sprint #30, DataGrid Sprint #33, DataGrid Sprint #34, DataGrid Sprint #35, DataGrid Sprint #36, DataGrid Sprint #37, DataGrid Sprint #38, DataGrid Sprint #39

    Description

      We would love to get our thread pools down to a single CPU thread pool (size = numCores) and a blocking thread pool (arbitrarily large). We may also require a scheduler pool for various options as well (limited size 1-2?).

      To do this we need to remove remnants of our blocking code as possible. Possible issues for blocking are mostly around locks and io operations.

      The persistence layer was completed with ISPN-9722 so that is not an issue.

      The requirement around locking can be relaxed if the locks are guaranteed to be small in scope and do not wrap other blocking operations. An example would be a lock such as ones in CHM as long as we don't have large blocks for functional argument types.

      If code cannot be made non blocking we must offload the operation to the blocking thread pool. Care must be taken to ensure that once the blocking portion of code is completed that we switch back the to CPU thread pool as soon as possible. The listener API for example is violating this and will run code in Infinispan from any thread that completes the listener that could be done from a user.

      Attachments

        Issue Links

          1.
          State Transfer needs to be made non blocking Sub-task Closed Major Dan Berindei
          2.
          Remove all the blocking Transport methods Sub-task Open Major Will Burns
          3.
          We should ensure that when CompletionStages are completed they are continued only on Infinispan threads Sub-task Resolved Major Will Burns
          4.
          Cache creation should be non blocking Sub-task New Major Will Burns
          5.
          Evaluate locations that use CompletableFuture#join Sub-task Coding In Progress Major Will Burns
          6.
          Non-blocking PendingLockManager Sub-task Closed Major Dan Berindei
          7.
          ExpirationManagerImpl Iteration removes from store synchronously Sub-task Closed Major Will Burns
          8.
          Consolidate Thread Pools Sub-task New Major Unassigned
          9.
          Make sure code is not using random executors or fork join pool Sub-task Open Major Will Burns
          10.
          Use stub for PersistenceManager when no stores are configured Sub-task Closed Major Will Burns
          11.
          Refactor ProtocolServer thread pools Sub-task New Major Unassigned
          12.
          Add checkstyle that prevents use of *Async with no executor from CompletionStage Sub-task Closed Major Will Burns
          13.
          Scattered state transfer should be non blocking Sub-task New Major Unassigned
          14.
          Convert usages of size to use sizeAsync Sub-task Closed Major Will Burns
          15.
          Add BlockHound to test suite to make sure cpu threads are never blocked Sub-task Closed Major Will Burns
          16.
          Add checkstyle to prevent invoking blocking methods from Flowable Sub-task Closed Major Will Burns
          17.
          Replace CompletionStageToPublisher with CompletionStageToSingle Sub-task Closed Major Will Burns
          18.
          Add interface to designate a thread is non blocking Sub-task Closed Major Will Burns
          19.
          Convert L1 to be non blocking Sub-task New Major Unassigned
          20.
          GlobalConfigurationManager should use blocking threads when starting caches Sub-task Closed Major Will Burns
          21.
          XSiteStateTransferProviderImpl is blocking when sending state Sub-task Closed Major Will Burns
          22.
          Conflict Manager blocks while doing processing Sub-task New Major Unassigned
          23.
          Convert async transport executor to the non blocking thread pool Sub-task Closed Major Will Burns
          24.
          Add in (non)blocking thread pools Sub-task Closed Major Will Burns
          25.
          InfinispanLock needs to block on a blocking thread but continue its operation on a non blocking thread Sub-task Closed Major Will Burns
          26.
          Convert Remote Command Executor to Non blocking/blocking thread executor Sub-task Closed Major Will Burns
          27.
          Requesting a cache that is starting will block Sub-task New Major Unassigned
          28.
          BasicComponentRegistryImpl can block when starting a component Sub-task Closed Major Will Burns
          29.
          Counter creation should be non blocking Sub-task New Major Will Burns
          30.
          Add blockhound support to hotrod server Sub-task Closed Major Will Burns
          31.
          PersistenceManagerImpl thread checks need to be updated Sub-task Closed Major Will Burns
          32.
          SegmentedBoundedOffHeapDataContainer#ensureSize should be marked as okay to be blocking Sub-task Closed Major Will Burns
          33.
          TransactionTable#calculateMinTopologyId should allow blocking Sub-task Closed Major Will Burns
          34.
          InvocationHelper should commit or rollback the transaction on a blocking thread Sub-task Closed Major Will Burns
          35.
          RecoveryManager API should be non blocking Sub-task Closed Major Will Burns
          36.
          Provide way for ClusterExecutor to use non blocking thread Sub-task Open Major Will Burns
          37.
          Change all commands to not set canBlock Sub-task New Major Will Burns
          38.
          TransactionCoordinator updated for non blocking Sub-task Closed Major Will Burns
          39.
          Convert detection of blocking or non blocking threads Sub-task Closed Major Will Burns
          40.
          PersistenceManagerImpl locks should be able to block Sub-task Closed Major Unassigned
          41.
          HotRod Server transaction is blocking Sub-task Closed Major Will Burns
          42.
          Replace HotRod worker thread pool Sub-task Closed Major Will Burns
          43.
          Convert ClientListenerRegistry to use non blocking calls Sub-task Closed Major Will Burns
          44.
          Add BlockingHandler to simplify running blocking operations Sub-task Closed Major Will Burns
          45.
          Add checkstyle to prevent invoking parallelStream Sub-task New Major Will Burns
          46.
          Cluster Locks should be non blocking Sub-task Resolved Major Unassigned
          47.
          Remove extra thread in BlockingTaskAwareExecutorServiceImpl Sub-task Closed Major Will Burns
          48.
          Fix blocking LimitedExecutor#addPermit test failures Sub-task Closed Major Will Burns

          Activity

            People

              wburns@redhat.com Will Burns
              wburns@redhat.com Will Burns
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: