Unfortunately we need 2 phase commit to guarantee consistency. Imagine the case where one of the owners says the write is okay and another says no, there is no way without two phase commit to guarantee that either all or none of the writes are completed.

One possibility would be node to deny a write if it expects it would result in other nodes running out of memory. However, this could still fail if some keys store more data than others. It would require Infinispan to calculate some probabilistic method of deciding when a node would run out of memory.

Another way would be to have a local (or shared) persistent store attached. In that case, if a backup owner will run out of memory if storing data, it would not store it in memory but store it to the persistent layer. If the node is restarted with new memory settings, the persistent stores would be consistent and the rebalance would put the data back in memory.