-
Bug
-
Resolution: Done
-
Normal
-
4.9
-
?
-
Moderate
-
None
-
Unspecified
-
None
-
If docs needed, set a value
Description of problem:
loadBalancerSource service spec is not enforcing the access restrictions to the LoadBalancer type service.
The LB is being created in OSP but it's reachable from the ranges not specified in loadBalancerSource.
Version-Release number of selected component (if applicable):
OCP 4.9.0-0.nightly-2021-07-07-021823
OSP 16.1.6 GA (RHOS-16.1-RHEL-8-20210604.n.0)
How reproducible: always
Steps to Reproduce:
1. Install OCP 4.9 on OSP (it can be either 13 or 16)
2. Enable the use of octavia in OCP (it can take ~20min):
oc edit cm cloud-provider-config -n openshift-config
- Add:
[LoadBalancer]
use-octavia = true
lb-provider = amphora
manage-security-groups = true
3. Create a ns and two pods for accessing the service later on:
oc new-project test1-ns
oc run --image quay.io/kuryr/demo test1-acceptcaller
oc run --image quay.io/kuryr/demo test1-dropcaller
oc get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
test1-acceptcaller 1/1 Running 0 13s 10.131.0.44 ostest-xlqbv-worker-0-szddn <none> <none>
test1-dropcaller 1/1 Running 0 26s 10.128.2.41 ostest-xlqbv-worker-0-lnrlx <none> <none>
4. Create a ns and a deployment:
oc new-project test2-ns
oc create deployment test2-lb --image=quay.io/kuryr/demo
5. Create a LoadBalancer type svc (setting the loadBalancerSourceRanges with the acceptcaller IP address):
cat test_range_svc.yaml
apiVersion: v1
items:
- apiVersion: v1
kind: Service
metadata:
finalizers: - service.kubernetes.io/load-balancer-cleanup
labels:
app: test2-lb
name: test2-range-svc
namespace: test2-ns
spec:
ipFamilies: - IPv4
ipFamilyPolicy: SingleStack
loadBalancerSourceRanges: - 10.131.0.44/32
ports: - port: 80
protocol: TCP
targetPort: 8080
selector:
app: test2-lb
sessionAffinity: None
type: LoadBalancer
kind: List
metadata:
resourceVersion: ""
selfLink: ""
oc -n test2-ns apply -f test_range_svc.yaml
- Wait until the load balancer is created and a fip assigned to it
openstack loadbalancer list
--------------------------------------------------------------------------------------------------------------------------------------------+
id | name | project_id | vip_address | provisioning_status | provider |
--------------------------------------------------------------------------------------------------------------------------------------------+
ee79d4f7-4b4c-426b-8516-690b5f5cefbb | a29ed801486034d279c8adc7a61e317c | c0316b3530e64b909f9451a857b404d0 | 10.196.1.83 | ACTIVE | amphora |
--------------------------------------------------------------------------------------------------------------------------------------------+
oc get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
test2-range-svc LoadBalancer 172.30.206.3 10.0.0.193 80:30117/TCP 2m11s
6. Check connectivity from acceptcaller:
oc -n test1-ns rsh test1-acceptcaller curl 10.0.0.193
test2-lb-5db6ffb54d-nt7ck: HELLO! I AM ALIVE!!!
7. Check connectivity from dropcaller (there shouldn't be connectivity):
oc -n test1-ns rsh test1-dropcaller curl 10.0.0.193
test2-lb-5db6ffb54d-nt7ck: HELLO! I AM ALIVE!!!
Actual results:
oc -n test1-ns rsh test1-dropcaller curl 10.0.0.193
test2-lb-5db6ffb54d-nt7ck: HELLO! I AM ALIVE!!!
Expected results: no connectivity
Additional info:
oc describe svc
Name: test2-range-svc
Namespace: test2-ns
Labels: app=test2-lb
Annotations: <none>
Selector: app=test2-lb
Type: LoadBalancer
IP Family Policy: SingleStack
IP Families: IPv4
IP: 172.30.206.3
IPs: 172.30.206.3
LoadBalancer Ingress: 10.0.0.193
Port: <unset> 80/TCP
TargetPort: 8080/TCP
NodePort: <unset> 30117/TCP
Endpoints: 10.128.2.42:8080
Session Affinity: None
External Traffic Policy: Cluster
LoadBalancer Source Ranges: 10.131.0.44/32
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning SyncLoadBalancerFailed 4m3s service-controller Error syncing load balancer: failed to ensure load balancer: Error reconciling security groups for LB service test2-ns/test2-range-svc: Error occurred updating port 6571a08b-81ad-4821-8d24-b869929ff3d1 for loadbalancer service test2-ns/test2-range-svc: Resource not found
Normal EnsuringLoadBalancer 3m58s (x2 over 6m1s) service-controller Ensuring load balancer
Normal EnsuredLoadBalancer 3m56s service-controller Ensured load balancer
The LB port in OSP is not being found.
openstack port list | grep 6571a08b
6571a08b-81ad-4821-8d24-b869929ff3d1 | octavia-lb-ee79d4f7-4b4c-426b-8516-690b5f5cefbb | fa:16:3e:44:62:0f | ip_address='10.196.1.83', subnet_id='c16b3b93-0cf0-4ec3-882c-1c2660e97889' | DOWN |
oc get svc -o yaml
apiVersion: v1
items:
- apiVersion: v1
kind: Service
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"finalizers":["service.kubernetes.io/load-balancer-cleanup"],"labels": {"app":"test2-lb"},"name":"test2-range-svc","namespace":"test2-ns"},"spec":{"ipFamilies":["IPv4"],"ipFamilyPolicy":"SingleStack","loadBalancerSourceRanges":["10.131.0.44/32"],"ports":[
{"port":80,"protocol":"TCP","targetPort":8080}],"selector":
{"app":"test2-lb"},"sessionAffinity":"None","type":"LoadBalancer"}}
creationTimestamp: "2021-07-09T12:24:52Z"
finalizers: - service.kubernetes.io/load-balancer-cleanup
labels:
app: test2-lb
name: test2-range-svc
namespace: test2-ns
resourceVersion: "605684"
uid: 29ed8014-8603-4d27-9c8a-dc7a61e317cf
spec:
clusterIP: 172.30.206.3
clusterIPs: - 172.30.206.3
externalTrafficPolicy: Cluster
ipFamilies: - IPv4
ipFamilyPolicy: SingleStack
loadBalancerSourceRanges: - 10.131.0.44/32
ports: - nodePort: 30117
port: 80
protocol: TCP
targetPort: 8080
selector:
app: test2-lb
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress: - ip: 10.0.0.193
kind: List
metadata:
resourceVersion: ""
selfLink: ""
Note that if manage-security-groups is not configured the LB creation will fail:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal EnsuringLoadBalancer 35s (x4 over 72s) service-controller Ensuring load balancer
Warning SyncLoadBalancerFailed 35s (x4 over 71s) service-controller Error syncing load balancer: failed to ensure load balancer: source range restrictions are not supported for openstack load balancers without managing security groups