Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-19863

AWS Security Group annotation for NLB type SVC not working

XMLWordPrintable

    • Critical
    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      The inbound rule gets created in default master and worker security group over AWS whenever a LB type SVC is created. In the customer's cluster where a huge number of LB SVCs are present, the inbound rule limit for security group reached due to which the following error comes up.
      
      "error sync Loadbalancer 'rules per security limit exceeded'"
      
      So we tried the below annotation for use custom security group but the annotation get ignored.
      --> service.beta.kubernetes.io/aws-load-balancer-extra-security-groups: sg-xxxx
      
      Further, we tested the below annotation to disable the automatic rule creation, which will avoid creating rules in default security groups and later on we can manually add the rule in custom security group.
      --> service.beta.kubernetes.io/aws-load-balancer-manage-backend-security-group-rules: "false"
      
      However, this annotation also gets ignored and we get same error. I found the following GitHub link for annotations to use.
      --> https://github.com/openshift/aws-load-balancer-controller/blob/54f88971f8ac73b07718d56d2ab4ba1f09ddebf4/docs/guide/service/annotations.md#annotations

      Version-Release number of selected component (if applicable):

       

      How reproducible:

      Everytime

      Steps to Reproduce:

      1. Deploy OCP 4 AWS IPI cluster.
      2. Create the NLB SVC with annotation.
      3. Sample SVC YAML.
      kind: Service
      apiVersion: v1
      metadata:
        annotations:
          service.beta.kubernetes.io/aws-load-balancer-manage-backend-security-group-rules: "false"
          service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: TCP
          service.beta.kubernetes.io/aws-load-balancer-internal: 'true'
          service.beta.kubernetes.io/aws-load-balancer-type: nlb
        name: nlb-with-sg
        labels:
          app: httpd-24
          app.kubernetes.io/component: httpd-24
          app.kubernetes.io/instance: httpd-24
      spec:
        externalTrafficPolicy: Cluster
        ipFamilies:
          - IPv4
        ports:
          - name: 8080-tcp
            protocol: TCP
            port: 8080
            targetPort: 24220
        internalTrafficPolicy: Cluster
        allocateLoadBalancerNodePorts: true
        type: LoadBalancer
        ipFamilyPolicy: SingleStack
        sessionAffinity: None
        selector:
          deployment: httpd-24
      
      4. Still the inbound rule gets created in master and worker security groups for NLB

              joelspeed Joel Speed
              rhn-support-aygarg Ayush Garg
              Huali Liu Huali Liu
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: