Most of the RPC timeout tasks are cancelled way before their timeout expires. This means the scheduler spends a lot of time reordering the elements of its DelayQueue.

It should be possible to store the tasks with a long timeout (e.g. 1s) in a queue and only move them to the scheduler's priority queue when they have less than 1s to expiration (e.g. by a worker thread that runs every 0.5s)

Storing all the tasks in a single queue may be impractical because the worker thread would have more and work to do as load increases and the RPCs take longer to finish, so a hashed timing wheel may be needed.