-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
4.17
-
None
-
Moderate
-
No
-
Rejected
-
False
-
Description of problem:
The recent rebase of Azure CCM https://github.com/openshift/cloud-provider-azure/pull/113 brought in https://github.com/kubernetes-sigs/cloud-provider-azure/pull/5885 which the Network Edge team has identified as causing some breakage in our E2E testing.
#5885 properly rejects both spec.LoadBalancerSourceRanges and "service.beta.kubernetes.io/load-balancer-source-ranges" set at the same time. It doesn't reconcile the service, nor does it allow for it's deletion. You will receive this error:
E0612 16:22:37.193349 1 azure_loadbalancer.go:2952] "Failed to parse access control configuration for service" err="cannot set both spec.LoadBalancerSourceRanges and service annotation service.beta.kubernetes.io/azure-allowed-ip-ranges" logger="reconcileSecurityGroup" cluster="ci-op-mv04309s-04a70-6jszn" service="openshift-ingress/router-sourcerangesstatus" load-balancer="ci-op-mv04309s-04a70-6jszn" delete-lb=true
Our E2E test intentionally validates both this annotation and spec set at the same time. However, to my knowledge, it's not a customer-driven requirement to support both, but instead an edge case that we ensured we tested for.
#5885 may be considered a incompatible change, and the cloud team should review whether they should:
- Set upgradeable=false based on the presence of the annotation & spec to 4.17
- Revert the Azure CCM #5885 so it continues to reconcile load balancers with the annotation and spec
- Do nothing
Version-Release number of selected component (if applicable):
4.17
How reproducible:
100%
Steps to Reproduce:
1. Create a service with both service.beta.kubernetes.io/load-balancer-source-ranges annotation and spec.LoadBalancerSourceRanges
Actual results:
In 4.17, the Service isn't reconciled
Expected results:
The cloud team should consider compatibility
Additional info:
A related PR, https://github.com/kubernetes-sigs/cloud-provider-azure/pull/5164, in an earlier rebase, which introduced the first restriction:
if len(sourceRanges) > 0 && len(allowedIPRanges) > 0 { logger.Error(err, "Forbidden configuration") return nil, ErrSetBothLoadBalancerSourceRangesAndAllowedIPRanges }
However that SourceRanges function providing the sourceRanges variable used spec.LoadBalancerSourceRanges OR
"service.beta.kubernetes.io/load-balancer-source-ranges", while the AllowedIPRanges function only used
"service.beta.kubernetes.io/azure-allowed-ip-ranges". So, that resulted in the logic:
# If result is false then either not valid or no source ranges provided:
(spec.LoadBalancerSourceRanges OR service.beta.kubernetes.io/load-balancer-source-ranges) XOR (service.beta.kubernetes.io/azure-allowed-ip-ranges)
That probably could be a breaking change too, but we don't use service.beta.kubernetes.io/azure-allowed-ip-ranges in our E2E tests, so we didn't notice that.
The change in https://github.com/kubernetes-sigs/cloud-provider-azure/pull/5885 reorganized the SourceRanges function by making it only use spec.LoadBalancerSourceRanges, and AllowedIPRanges now uses both of the annotations. This resulted in:
# If result is false then either not valid or no source ranges provided:
(spec.LoadBalancerSourceRanges) XOR (service.beta.kubernetes.io/load-balancer-source-ranges OR service.beta.kubernetes.io/azure-allowed-ip-ranges)
- relates to
-
OCPBUGS-35445 TestAllowedSourceRangesStatus doesn't clean up service object
-
- New
-
- links to