There's a hole in state transfer mechanism that can occur when a node is leaving the cluster, but it was creating the entries and was only able to replicate the data to some of the nodes.

The problem occurs when the segment ownership of the node doesn't change after the rebalance. Since state transfer does not request state for keys in which it is already an owner, the cache could be left in a state where a key is resident < numOwners nodes. In addition, this could be any subset of the primary OR backup nodes.