The recent rebase of Azure CCM https://github.com/openshift/cloud-provider-azure/pull/113 brought in https://github.com/kubernetes-sigs/cloud-provider-azure/pull/5885 which the Network Edge team has identified as causing some breakage in our E2E testing.

#5885 properly rejects both spec.LoadBalancerSourceRanges and "service.beta.kubernetes.io/load-balancer-source-ranges" set at the same time. It doesn't reconcile the service, nor does it allow for it's deletion. You will receive this error:

E0612 16:22:37.193349       1 azure_loadbalancer.go:2952] "Failed to parse access control configuration for service" err="cannot set both spec.LoadBalancerSourceRanges and service annotation service.beta.kubernetes.io/azure-allowed-ip-ranges" logger="reconcileSecurityGroup" cluster="ci-op-mv04309s-04a70-6jszn" service="openshift-ingress/router-sourcerangesstatus" load-balancer="ci-op-mv04309s-04a70-6jszn" delete-lb=true

Our E2E test intentionally validates both this annotation and spec set at the same time. However, to my knowledge, it's not a customer-driven requirement to support both, but instead an edge case that we ensured we tested for.

#5885 may be considered a incompatible change, and the cloud team should review whether they should:

1. Set upgradeable=false based on the presence of the annotation & spec to 4.17
2. Revert the Azure CCM #5885 so it continues to reconcile load balancers with the annotation and spec
3. Do nothing

A related PR, https://github.com/kubernetes-sigs/cloud-provider-azure/pull/5164, in an earlier rebase, which introduced the first restriction: 

    if len(sourceRanges) > 0 && len(allowedIPRanges) > 0 {
        logger.Error(err, "Forbidden configuration")
        return nil, ErrSetBothLoadBalancerSourceRangesAndAllowedIPRanges
   }

However that SourceRanges function providing the sourceRanges variable used spec.LoadBalancerSourceRanges OR 
"service.beta.kubernetes.io/load-balancer-source-ranges", while the AllowedIPRanges function only used 
"service.beta.kubernetes.io/azure-allowed-ip-ranges". So, that resulted in the logic:

# If result is false then either not valid or no source ranges provided:
(spec.LoadBalancerSourceRanges OR service.beta.kubernetes.io/load-balancer-source-ranges) XOR (service.beta.kubernetes.io/azure-allowed-ip-ranges)

That probably could be a breaking change too, but we don't use service.beta.kubernetes.io/azure-allowed-ip-ranges in our E2E tests, so we didn't notice that.
 
The change in https://github.com/kubernetes-sigs/cloud-provider-azure/pull/5885 reorganized the SourceRanges function by making it only use spec.LoadBalancerSourceRanges, and AllowedIPRanges now uses both of the annotations. This resulted in:

# If result is false then either not valid or no source ranges provided:
(spec.LoadBalancerSourceRanges) XOR (service.beta.kubernetes.io/load-balancer-source-ranges OR service.beta.kubernetes.io/azure-allowed-ip-ranges)