Refactoring in ETCD-512 does not clean up certificates that are dynamically generated. Imagine you're recreating all your master nodes everyday, we would create new peer/serving/metrics certificates for each node and never clean them up.

We should try to be conservative when cleaning them up, so keep them around for a certain retention period (7-10 days?) after the node went away.

AC:

• CEO should clean up old-enough "node" certificates