Currently, session affinity is specified as a single route, usually the primary owner of a session. However, if this node is not active (from the load balancer's perspective), then the load balancer will select another node of its choosing. This is less than ideal, since some nodes are more optimal than others. Better is to expression session affinity as an ordered list of nodes. The load balancer can then choose the 1st node in the list that is active, and cascade on failures accordingly. From the server's perspective, the affinity list should have the following order:
1. The primary owner
2. Any backup owners
3. The local node (if the local node is not an owner)
There are a couple of ways a load balancer can indicate that it supports multiple routes.
1. Static configuration, e.g. routing="NONE|OWNER|LOCAL|RANKED"
2. Adding an HTTP header to a proxied request, which can be detected by the server handling the request
#1 has the advantage of simplicity - but requires that users manually apply this configuration to gain optimal performance. This would also let us allow users (for even the single node affinity case) to toggle between local affinity or primary owner affinity, or even no routing.
#2 has the advantage of applying this optimization automatically, but incurs the cost of an additional header to every request