-
Enhancement
-
Resolution: Unresolved
-
Major
-
None
-
None
-
False
-
-
False
-
-
In DBZ-8924 we introduced the new unbuffered LogMiner implementation. This implementation relies on secondary LogMiner session that recomputes the low watermark read position periodically. This computation can be intensive and could take several minutes to complete depending on the volume of activity since the current resume position.
I'd like to propose we move this calculation to a secondary thread, introduce some state machine aspects to avoid race conditions, allowing the main thread to request a recalculation on the given interval, and only starting it if its not in-progress, and only providing the updated information to the main thread when requested.
So in practice there would be a few states:
- Ready
This state signals that if the timer has expired, the resume position will trigger the background thread to compute the value. The handler will transition into an Inprogress state until the background thread computes the value. - Inprogress
This state means the background thread is running, and any call to the handler should continue to return the current resume position, as if the timer has not yet expired. - Waiting
This state means that the background thread has computed the resume position and is waiting on the main thread to call the handler. When the main thread does, it will return the computed value, resets the timer, and transitions back into the Ready state.
This has various benefits, but primarily avoids blocking the main change event stream while the recalculation step runs. But this will introduce slightly more load on the database given that we have 2 LogMiner sessions that are running concurrently per connector.
- depends on
-
DBZ-8924 Implement LogMiner committed data only unbuffered adapter
-
- Closed
-