As a developer of TNF, I need:

• To be able to recover via fencing from ungraceful shutdown events
• To be able to validate that fencing has been set up correctly by triggering an in-cluster recovery test

Acceptance Criteria

• Ungraceful shutdowns events or fencing result in one node assuming full control by fencing the other node, forcing a restart of etcd as a cluster of one, and restarting the other node so that it can resync and rejoin etcd
• Tests are added to cover scenarios that trigger fencing and ungraceful shutdown events
• In cluster verification exists that the customer can run to try exercise the fencing and node recovery setup

TNF Enhancement Proposal