Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-3700

[osp][octavia lb] NodePort allocation cannot be disabled for LB type svcs

    XMLWordPrintable

Details

    • +
    • Low
    • False
    • Hide

      None

      Show
      None
    • None

    Description

      Description of problem:

      Node port allocation [1] cannot be disabled in LoadBalancer type services. CCM will prevent LB creation because the `ProtocolPort` parameter is missing:
      "Missing input for argument [ProtocolPort]"

      Version-Release number of selected component (if applicable):

      4.12.0-0.nightly-2022-08-30-054458

      How reproducible:

      Always

      Steps to Reproduce ([^no-nodeport-manifests.yaml]):

      1. Create the svc with allocateLoadBalancerNodePorts: false
      
      $ oc create -f no-nodeport-manifests.yaml
      project.project.openshift.io/lb-no-nodeport-ns created
      deployment.apps/lb-no-nodeport-dep created                              
      service/lb-no-nodeport-svc created
      
      2. Check svc and LB creation

      Actual results:

      $ oc -n lb-no-nodeport-ns describe svc lb-no-nodeport-svc
      [...]
      Events:
        Type     Reason                  Age                    From                Message
        ----     ------                  ----                   ----                -------
        Warning  SyncLoadBalancerFailed  3m17s (x4 over 3m53s)  service-controller  Error syncing load balancer: failed to ensure load balancer: error creating loadbalancer kube_service_kubernetes_lb-no-nodeport-ns_lb-no-nodeport-svc: error creating loadbalancer {"name":"kube_service_kubernetes_lb-no-nodeport-ns_lb-no-nodeport-svc","description":"Kubernetes external service lb-no-nodeport-ns/lb-no-nodeport-svc from cluster kubernetes","vip_subnet_id":"cda25f0b-5dcf-49be-801e-ce5a74aee027","provider":"amphora","listeners":[{"protocol":"TCP","protocol_port":80,"name":"listener_0_kube_service_kubernetes_lb-no-nodeport-ns_lb-no-nodeport-svc","default_pool":{"lb_algorithm":"ROUND_ROBIN","protocol":"TCP","name":"pool_0_kube_service_kubernetes_lb-no-nodeport-ns_lb-no-nodeport-svc","members":[{"address":"10.196.1.207","protocol_port":0,"name":"ostest-55d8f-master-2","subnet_id":"cda25f0b-5dcf-49be-801e-ce5a74aee027"},{"address":"10.196.0.131","protocol_port":0,"name":"ostest-55d8f-master-1","subnet_id":"cda25f0b-5dcf-49be-801e-ce5a74aee027"},{"address":"10.196.1.42","protocol_port":0,"name":"ostest-55d8f-master-0","subnet_id":"cda25f0b-5dcf-49be-801e-ce5a74aee027"},{"address":"10.196.0.103","protocol_port":0,"name":"ostest-55d8f-worker-0-6rw6v","subnet_id":"cda25f0b-5dcf-49be-801e-ce5a74aee027"},{"address":"10.196.0.222","protocol_port":0,"name":"ostest-55d8f-worker-0-q42h6","subnet_id":"cda25f0b-5dcf-49be-801e-ce5a74aee027"}]},"connection_limit":-1,"timeout_client_data":50000,"timeout_member_data":50000,"timeout_member_connect":5000,"timeout_tcp_inspect":0,"allowed_cidrs":["0.0.0.0/0"],"tags":["kube_service_kubernetes_lb-no-nodeport-ns_lb-no-nodeport-svc"]}],"tags":["kube_service_kubernetes_lb-no-nodeport-ns_lb-no-nodeport-svc"]}: Missing input for argument [ProtocolPort]
        Warning  SyncLoadBalancerFailed  2m36s                  service-controller  Error syncing load balancer: failed to ensure load balancer: error creating loadbalancer kube_service_kubernetes_lb-no-nodeport-ns_lb-no-nodeport-svc: error creating loadbalancer {"name":"kube_service_kubernetes_lb-no-nodeport-ns_lb-no-nodeport-svc","description":"Kubernetes external service lb-no-nodeport-ns/lb-no-nodeport-svc from cluster kubernetes","vip_subnet_id":"cda25f0b-5dcf-49be-801e-ce5a74aee027","provider":"amphora","listeners":[{"protocol":"TCP","protocol_port":80,"name":"listener_0_kube_service_kubernetes_lb-no-nodeport-ns_lb-no-nodeport-svc","default_pool":{"lb_algorithm":"ROUND_ROBIN","protocol":"TCP","name":"pool_0_kube_service_kubernetes_lb-no-nodeport-ns_lb-no-nodeport-svc","members":[{"address":"10.196.0.222","protocol_port":0,"name":"ostest-55d8f-worker-0-q42h6","subnet_id":"cda25f0b-5dcf-49be-801e-ce5a74aee027"},{"address":"10.196.1.207","protocol_port":0,"name":"ostest-55d8f-master-2","subnet_id":"cda25f0b-5dcf-49be-801e-ce5a74aee027"},{"address":"10.196.0.131","protocol_port":0,"name":"ostest-55d8f-master-1","subnet_id":"cda25f0b-5dcf-49be-801e-ce5a74aee027"},{"address":"10.196.1.42","protocol_port":0,"name":"ostest-55d8f-master-0","subnet_id":"cda25f0b-5dcf-49be-801e-ce5a74aee027"},{"address":"10.196.0.103","protocol_port":0,"name":"ostest-55d8f-worker-0-6rw6v","subnet_id":"cda25f0b-5dcf-49be-801e-ce5a74aee027"}]},"connection_limit":-1,"timeout_client_data":50000,"timeout_member_data":50000,"timeout_member_connect":5000,"timeout_tcp_inspect":0,"allowed_cidrs":["0.0.0.0/0"],"tags":["kube_service_kubernetes_lb-no-nodeport-ns_lb-no-nodeport-svc"]}],"tags":["kube_service_kubernetes_lb-no-nodeport-ns_lb-no-nodeport-svc"]}: Missing input for argument [ProtocolPort] 
      
      $ oc -n lb-no-nodeport-ns get svc
      NAME                 TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
      lb-no-nodeport-svc   LoadBalancer   172.30.153.11   <pending>     80/TCP    5m58s
      
      $ oc -n lb-no-nodeport-ns get svc lb-no-nodeport-svc -o yaml
      apiVersion: v1
      kind: Service
      metadata:
        creationTimestamp: "2022-09-06T08:08:06Z"
        finalizers:
        - service.kubernetes.io/load-balancer-cleanup
        labels:
          app: lb-no-nodeport-dep
        name: lb-no-nodeport-svc
        namespace: lb-no-nodeport-ns
        resourceVersion: "3621284"
        uid: f7574fd8-c6f8-4950-b7ad-00e5136ba291
      spec:
        allocateLoadBalancerNodePorts: false
        clusterIP: 172.30.153.11
        clusterIPs:
        - 172.30.153.11
        externalTrafficPolicy: Cluster
        internalTrafficPolicy: Cluster
        ipFamilies:
        - IPv4
        ipFamilyPolicy: SingleStack
        ports:
        - port: 80
          protocol: TCP
          targetPort: 8080
        selector:
          app: lb-no-nodeport-dep
        sessionAffinity: None
        type: LoadBalancer
      status:
        loadBalancer: {}
      
      $ openstack loadbalancer list
      (empty)

      Expected results:

      LB should be created and a FIP assigned to the svc although the connectivity from the LB to the endpoint pods (without NodePort) is not expected to work.

      Additional info:

      This issue has been discovered when running a k8s conformance test [2], after enabling it for openstack platform (but it's not expected to pass).

      [1]https://kubernetes.io/docs/concepts/services-networking/service/#load-balancer-nodeport-allocation

      [2]https://github.com/openshift/origin/blob/54548ade0f7bd86c00d9aa9e91f51ed51f32bea1/vendor/k8s.io/kubernetes/test/e2e/network/loadbalancer.go#L855

      Attachments

        Issue Links

          Activity

            People

              mdulko Michał Dulko
              juriarte@redhat.com Jon Uriarte
              Ramón Lobillo Ramón Lobillo
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: