We know from experience with RHQ/JON that we want to support configurable data retentions. The TTL is set per cell and adds four bytes of overhead per cell. If the user changes the data retention settings, we have to read every data point and re-insert it with the new TTL. This would have to be done for all effected metrics. This will be very expensive in terms of I/O.
An alternative approach is to perform partition deletes based on the time buckets described in HWKMETRICS-189. Suppose we use bucket sizes of a day and a data retention of one week. We would have a job that runs daily and deletes the 8th oldest partition for each metric.
There are several advantages to this approach over using TTL. First, we do not incur the 4 bytes overhead per cell. Secondly and more importantly, changing data retention becomes much more efficient. Let's say we want to change the data retention in the previous example from one week to two weeks. We simply update the retention setting to reflect this and when the job runs it deletes the 15th oldest partition instead of the 8th.
There is one caveat with this approach. The retention cannot be smaller than the time bucket size. Furthermore, the retention needs to be evenly divisible by the bucket size. If the retention is smaller, then we cannot do partition-wide deletes. If the bucket size is a month, then the retention size will need to be specified in terms of months.