An ARO customer asked how to restrict end user application access by source IP. It's actually quite fiddly to do and I don't believe our 4.x documentation covers it nearly well enough, especially given that this is surely not an uncommon use case. This is the answer I sent back to the customer; it is somewhat Azure specific; it would be great if this could be used as a basis to significantly improve our documentation in this space.

==== 8< ====

My general recommendation here is: either use Routes/Ingress objects combined with the haproxy.router.openshift.io/ip_whitelist annotation, or use a Service object of type LoadBalancer with spec.externalTrafficPolicy set to Local, alongside a NetworkPolicy object. The rationale for this advice is that although spec.loadBalancerSourceRanges also works on a Service, overall the Azure quota of NetworkSecurityGroup (NSG) rules is low and can fairly easily be exhausted.

OpenShift Routes and Ingress
============================

• application sees original source address information in X-Forwarded- {For,Host,Port,Proto}

  HTTP headers (note that if other proxies are in use, multiple matching headers can be present in the request).

• last hop source address whitelisting can be applied using haproxy.router.openshift.io/ip_whitelist annotation on Route/Ingress [1].
• Setting NetworkPolicy ipBlock with last hop source address is not applicable because traffic is proxied by the OpenShift router component.
  [1] https://docs.openshift.com/container-platform/4.4/networking/routes/route-configuration.html#nw-route-specific-annotations_route-configuration

Kubernetes LoadBalancer Services (spec.externalTrafficPolicy: Cluster)
======================================================================

• application cannot see last hop source address information.
• NetworkPolicy / ipBlock cannot be used to restrict accesses (don't know if this is a bug or not, but it's definitely a gotcha).
• whitelisting can be applied using loadBalancerSourceRanges (this configures the NSG), but note that there is only one NSG per cluster and the maximum number of rules per NSG on Azure is low.

Kubernetes LoadBalancer Services (spec.externalTrafficPolicy: Local)
====================================================================

• application sees last hop source address information transparently.
• whitelisting can be applied using NetworkPolicy / ipBlock.
• whitelisting can also be applied using loadBalancerSourceRanges (this configures the NSG), but note that there is only one NSG per cluster and the maximum number of rules per NSG on Azure is low.