Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-8913

[osp][octavia lb] loadBalancerSourceRanges not supported in LoadBalancer type services on Openstack

XMLWordPrintable

    • ?
    • Moderate
    • Unspecified
    • None
    • If docs needed, set a value

      Description of problem:

      loadBalancerSource service spec is not enforcing the access restrictions to the LoadBalancer type service.
      The LB is being created in OSP but it's reachable from the ranges not specified in loadBalancerSource.

      Version-Release number of selected component (if applicable):
      OCP 4.9.0-0.nightly-2021-07-07-021823
      OSP 16.1.6 GA (RHOS-16.1-RHEL-8-20210604.n.0)

      How reproducible: always

      Steps to Reproduce:
      1. Install OCP 4.9 on OSP (it can be either 13 or 16)

      2. Enable the use of octavia in OCP (it can take ~20min):
      oc edit cm cloud-provider-config -n openshift-config

      1. Add:
        [LoadBalancer]
        use-octavia = true
        lb-provider = amphora
        manage-security-groups = true

      3. Create a ns and two pods for accessing the service later on:
      oc new-project test1-ns
      oc run --image quay.io/kuryr/demo test1-acceptcaller
      oc run --image quay.io/kuryr/demo test1-dropcaller

      oc get pods -o wide
      NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
      test1-acceptcaller 1/1 Running 0 13s 10.131.0.44 ostest-xlqbv-worker-0-szddn <none> <none>
      test1-dropcaller 1/1 Running 0 26s 10.128.2.41 ostest-xlqbv-worker-0-lnrlx <none> <none>

      4. Create a ns and a deployment:
      oc new-project test2-ns
      oc create deployment test2-lb --image=quay.io/kuryr/demo

      5. Create a LoadBalancer type svc (setting the loadBalancerSourceRanges with the acceptcaller IP address):
      cat test_range_svc.yaml
      apiVersion: v1
      items:

      • apiVersion: v1
        kind: Service
        metadata:
        finalizers:
      • service.kubernetes.io/load-balancer-cleanup
        labels:
        app: test2-lb
        name: test2-range-svc
        namespace: test2-ns
        spec:
        ipFamilies:
      • IPv4
        ipFamilyPolicy: SingleStack
        loadBalancerSourceRanges:
      • 10.131.0.44/32
        ports:
      • port: 80
        protocol: TCP
        targetPort: 8080
        selector:
        app: test2-lb
        sessionAffinity: None
        type: LoadBalancer
        kind: List
        metadata:
        resourceVersion: ""
        selfLink: ""

      oc -n test2-ns apply -f test_range_svc.yaml

      1. Wait until the load balancer is created and a fip assigned to it

      openstack loadbalancer list
      --------------------------------------------------------------------------------------------------------------------------------------------+

      id name project_id vip_address provisioning_status provider

      --------------------------------------------------------------------------------------------------------------------------------------------+

      ee79d4f7-4b4c-426b-8516-690b5f5cefbb a29ed801486034d279c8adc7a61e317c c0316b3530e64b909f9451a857b404d0 10.196.1.83 ACTIVE amphora

      --------------------------------------------------------------------------------------------------------------------------------------------+

      oc get svc
      NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
      test2-range-svc LoadBalancer 172.30.206.3 10.0.0.193 80:30117/TCP 2m11s

      6. Check connectivity from acceptcaller:
      oc -n test1-ns rsh test1-acceptcaller curl 10.0.0.193
      test2-lb-5db6ffb54d-nt7ck: HELLO! I AM ALIVE!!!

      7. Check connectivity from dropcaller (there shouldn't be connectivity):
      oc -n test1-ns rsh test1-dropcaller curl 10.0.0.193
      test2-lb-5db6ffb54d-nt7ck: HELLO! I AM ALIVE!!!

      Actual results:
      oc -n test1-ns rsh test1-dropcaller curl 10.0.0.193
      test2-lb-5db6ffb54d-nt7ck: HELLO! I AM ALIVE!!!

      Expected results: no connectivity

      Additional info:
      oc describe svc
      Name: test2-range-svc
      Namespace: test2-ns
      Labels: app=test2-lb
      Annotations: <none>
      Selector: app=test2-lb
      Type: LoadBalancer
      IP Family Policy: SingleStack
      IP Families: IPv4
      IP: 172.30.206.3
      IPs: 172.30.206.3
      LoadBalancer Ingress: 10.0.0.193
      Port: <unset> 80/TCP
      TargetPort: 8080/TCP
      NodePort: <unset> 30117/TCP
      Endpoints: 10.128.2.42:8080
      Session Affinity: None
      External Traffic Policy: Cluster
      LoadBalancer Source Ranges: 10.131.0.44/32
      Events:
      Type Reason Age From Message
      ---- ------ ---- ---- -------
      Warning SyncLoadBalancerFailed 4m3s service-controller Error syncing load balancer: failed to ensure load balancer: Error reconciling security groups for LB service test2-ns/test2-range-svc: Error occurred updating port 6571a08b-81ad-4821-8d24-b869929ff3d1 for loadbalancer service test2-ns/test2-range-svc: Resource not found
      Normal EnsuringLoadBalancer 3m58s (x2 over 6m1s) service-controller Ensuring load balancer
      Normal EnsuredLoadBalancer 3m56s service-controller Ensured load balancer

      The LB port in OSP is not being found.

      openstack port list | grep 6571a08b

      6571a08b-81ad-4821-8d24-b869929ff3d1 octavia-lb-ee79d4f7-4b4c-426b-8516-690b5f5cefbb fa:16:3e:44:62:0f ip_address='10.196.1.83', subnet_id='c16b3b93-0cf0-4ec3-882c-1c2660e97889' DOWN

      oc get svc -o yaml
      apiVersion: v1
      items:

      • apiVersion: v1
        kind: Service
        metadata:
        annotations:
        kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"finalizers":["service.kubernetes.io/load-balancer-cleanup"],"labels": {"app":"test2-lb"}

        ,"name":"test2-range-svc","namespace":"test2-ns"},"spec":{"ipFamilies":["IPv4"],"ipFamilyPolicy":"SingleStack","loadBalancerSourceRanges":["10.131.0.44/32"],"ports":[

        {"port":80,"protocol":"TCP","targetPort":8080}

        ],"selector":

        {"app":"test2-lb"}

        ,"sessionAffinity":"None","type":"LoadBalancer"}}
        creationTimestamp: "2021-07-09T12:24:52Z"
        finalizers:

      • service.kubernetes.io/load-balancer-cleanup
        labels:
        app: test2-lb
        name: test2-range-svc
        namespace: test2-ns
        resourceVersion: "605684"
        uid: 29ed8014-8603-4d27-9c8a-dc7a61e317cf
        spec:
        clusterIP: 172.30.206.3
        clusterIPs:
      • 172.30.206.3
        externalTrafficPolicy: Cluster
        ipFamilies:
      • IPv4
        ipFamilyPolicy: SingleStack
        loadBalancerSourceRanges:
      • 10.131.0.44/32
        ports:
      • nodePort: 30117
        port: 80
        protocol: TCP
        targetPort: 8080
        selector:
        app: test2-lb
        sessionAffinity: None
        type: LoadBalancer
        status:
        loadBalancer:
        ingress:
      • ip: 10.0.0.193
        kind: List
        metadata:
        resourceVersion: ""
        selfLink: ""

      Note that if manage-security-groups is not configured the LB creation will fail:
      Events:
      Type Reason Age From Message
      ---- ------ ---- ---- -------
      Normal EnsuringLoadBalancer 35s (x4 over 72s) service-controller Ensuring load balancer
      Warning SyncLoadBalancerFailed 35s (x4 over 71s) service-controller Error syncing load balancer: failed to ensure load balancer: source range restrictions are not supported for openstack load balancers without managing security groups

            emacchi@redhat.com Emilien Macchi
            juriarte@redhat.com Jon Uriarte
            Jon Uriarte Jon Uriarte
            Red Hat Employee
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: