When we terminate a member (EC2's "terminate" function) or kill -9 it, then the file (or bucket data in S3) won't get removed. This leads to stale data. On EC2, I expect that virtualized instances are often simply terminated, so this problem is compounded there.

SOLUTION:

• Periodically write own data to the file system (FILE_PING) or S3 (S3_PING)
• On a view change: remove all data that's not in the current view