-
Bug
-
Resolution: Done
-
Major
-
0.7.1
-
None
Version: 0.7.1
Modules: postgres connector, EmbeddedEngine
Postgres Version: AWS RDS 9.6.5
There may be chance that LSN committed back to Postgres can be later than what written to offset store. task.commit() on postgres connector (commitRecord() is not implemented) seems to use latest lsn received (e.g. record it puts into queue). Offset (LSN) that is written to the store is the latest record consumed by the consumer which may not be the latest one from the queue.
I believe postgres may truncate / archive WAL block once LSN is committed?
To reproduce:
1) Make consumer (notify callback) slow so that changes are queue up
2) Start EmbeddedEngine
3) Make DB changes continuously until commit happens
4) Check offset store for LSN and postgres committed LSN