While implementing FUA support in file-posix for RHEL-66064, I noticed that unexpectedly, in some cases performance decreases instead of increasing when the additional fdatasync() calls aren't needed any more.

Looking closer at the problem, I noticed that enabling FUA caused polling to be too short to be effective any more. The sequence during a request with flush emulation looks like this:

1. Perform a write request on a slow disk -> polling disabled
2. Completion involves a BH -> polling enabled with 4 µs
3. Flush request -> fast because disk is write through, polling increased to 8 µs
4. Completion involves a BH -> polling stays enabled

This time was enough for fio in the guest to issue the next request so that polling in the host was effective. Without the flush request, polling duration remained too short and the iothread gave up.

Obviously, this is not a sensible way to tune the polling duration. Instead, we should distinguish between different events, record their latencies separately and apply the polling duration that matches those events that we expect to happen next.