RocksDB allows the application to process/delete entries during compaction, and we could use this CompactionFilter callback to implement the purging of expired entries.
It's not straightforward, through: CompactionFilter must be implemented in a native (C++) library. rocksdb-jni allows using a native implementation from Java, but it does not allow defining it in Java (like it does for comparators).
- Write a native library implementing CompactionFilter for our current storage format
- Change the value format to match the Cassandra storage format and use the predefined CassandraCompactionFilter
- Revive the now-removed LuaCompactionFilter and write our filter implementation in Lua
- Contribute an adapter for Java CompactionFilter implementations to the RocksDB project, inspired by the existing AbstractComparator