Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-35089

seeing "The ingress ip for service openshift-ingress/router-default is not in the ingress range" in disruptive test

XMLWordPrintable

    • Important
    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      seeing below message of the ingress LoadBalancer service during QE e2e testjob of profile gcp-upi-private-xpn-ingress-glb-amd-f28-destructive
      
        Warning  IngressIPReallocated  152m  ingressip-controller  The ingress ip 10.0.32.54 for service openshift-ingress/router-default is not in the ingress range.  A new ip will be allocated.

      Version-Release number of selected component (if applicable):

      4.14.0-0.nightly-multi-2024-06-06-222416

      How reproducible:

      just found in the disruptive test

      Steps to Reproduce:

      1. install GCP XPN private cluster
      2. run disruptive e2e test
      3.
      

      Actual results:

      the internal LB service is assigned public IP during the test
      
      $ oc -n openshift-ingress get svc
      NAME                      TYPE           CLUSTER-IP       EXTERNAL-IP    PORT(S)                      AGE
      router-default            LoadBalancer   172.30.196.162   34.170.143.3   80:32279/TCP,443:32073/TCP   9h
      router-internal-default   ClusterIP      172.30.134.52    <none>         80/TCP,443/TCP,1936/TCP      9h
      
      
      $ oc -n openshift-ingress get svc router-default -oyaml
      apiVersion: v1
      kind: Service
      metadata:
        annotations:
          cloud.google.com/load-balancer-type: Internal
          networking.gke.io/internal-load-balancer-allow-global-access: "true"
          traffic-policy.network.alpha.openshift.io/local-with-fallback: ""
        creationTimestamp: "2024-06-07T02:39:23Z"
        finalizers:
        - gke.networking.io/l4-ilb-v1
        - service.kubernetes.io/load-balancer-cleanup
        labels:
          app: router
          ingresscontroller.operator.openshift.io/owning-ingresscontroller: default
          router: router-default
        name: router-default
        namespace: openshift-ingress
        ownerReferences:
        - apiVersion: apps/v1
          controller: true
          kind: Deployment
          name: router-default
          uid: 0a93a445-1f35-4b5a-81c7-d5b869a444d0
        resourceVersion: "381089"
        uid: c4ce5983-2abf-4c29-a32e-425d1d4e4fe9
      spec:
        allocateLoadBalancerNodePorts: true
        clusterIP: 172.30.196.162
        clusterIPs:
        - 172.30.196.162
        externalTrafficPolicy: Local
        healthCheckNodePort: 32348
        internalTrafficPolicy: Cluster
        ipFamilies:
        - IPv4
        ipFamilyPolicy: SingleStack
        ports:
        - name: http
          nodePort: 32279
          port: 80
          protocol: TCP
          targetPort: http
        - name: https
          nodePort: 32073
          port: 443
          protocol: TCP
          targetPort: https
        selector:
          ingresscontroller.operator.openshift.io/deployment-ingresscontroller: default
        sessionAffinity: None
        type: LoadBalancer
      status:
        loadBalancer:
          ingress:
          - ip: 34.170.143.3
      
      
      $ oc -n openshift-ingress describe svc router-default
      Name:                     router-default
      Namespace:                openshift-ingress
      Labels:                   app=router
                                ingresscontroller.operator.openshift.io/owning-ingresscontroller=default
                                router=router-default
      Annotations:              cloud.google.com/load-balancer-type: Internal
                                networking.gke.io/internal-load-balancer-allow-global-access: true
                                traffic-policy.network.alpha.openshift.io/local-with-fallback: 
      Selector:                 ingresscontroller.operator.openshift.io/deployment-ingresscontroller=default
      Type:                     LoadBalancer
      IP Family Policy:         SingleStack
      IP Families:              IPv4
      IP:                       172.30.196.162
      IPs:                      172.30.196.162
      LoadBalancer Ingress:     34.170.143.3
      Port:                     http  80/TCP
      TargetPort:               http/TCP
      NodePort:                 http  32279/TCP
      Endpoints:                10.128.2.10:80,10.131.1.37:80
      Port:                     https  443/TCP
      TargetPort:               https/TCP
      NodePort:                 https  32073/TCP
      Endpoints:                10.128.2.10:443,10.131.1.37:443
      Session Affinity:         None
      External Traffic Policy:  Local
      HealthCheck NodePort:     32348
      Events:
        Type     Reason                Age   From                  Message
        ----     ------                ----  ----                  -------
        Warning  IngressIPReallocated  152m  ingressip-controller  The ingress ip 10.0.32.54 for service openshift-ingress/router-default is not in the ingress range.  A new ip will be allocated.
      
      
      

      Expected results:

      the LB should be provisioned and internal

      Additional info:

      after deleting the LB service and wait it was recreated, the new one looks good. ,e.g
      
      $ oc -n openshift-ingress get svc
      NAME                      TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
      router-default            LoadBalancer   172.30.12.54    <pending>     80:30353/TCP,443:31070/TCP   15s
      router-internal-default   ClusterIP      172.30.134.52   <none>        80/TCP,443/TCP,1936/TCP      10h
      
      $ oc -n openshift-ingress get svc
      NAME                      TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
      router-default            LoadBalancer   172.30.12.54    10.0.32.26    80:30353/TCP,443:31070/TCP   89s
      router-internal-default   ClusterIP      172.30.134.52   <none>        80/TCP,443/TCP,1936/TCP      10h
      
      $ oc -n openshift-ingress describe svc router-default
      Name:                     router-default
      Namespace:                openshift-ingress
      Labels:                   app=router
                                ingresscontroller.operator.openshift.io/owning-ingresscontroller=default
                                router=router-default
      Annotations:              cloud.google.com/load-balancer-type: Internal
                                networking.gke.io/internal-load-balancer-allow-global-access: true
                                traffic-policy.network.alpha.openshift.io/local-with-fallback: 
      Selector:                 ingresscontroller.operator.openshift.io/deployment-ingresscontroller=default
      Type:                     LoadBalancer
      IP Family Policy:         SingleStack
      IP Families:              IPv4
      IP:                       172.30.12.54
      IPs:                      172.30.12.54
      LoadBalancer Ingress:     10.0.32.26
      Port:                     http  80/TCP
      TargetPort:               http/TCP
      NodePort:                 http  30353/TCP
      Endpoints:                10.128.2.10:80,10.131.1.37:80
      Port:                     https  443/TCP
      TargetPort:               https/TCP
      NodePort:                 https  31070/TCP
      Endpoints:                10.128.2.10:443,10.131.1.37:443
      Session Affinity:         None
      External Traffic Policy:  Local
      HealthCheck NodePort:     30265
      Events:
        Type    Reason                    Age   From                Message
        ----    ------                    ----  ----                -------
        Normal  EnsuringLoadBalancer      103s  service-controller  Ensuring load balancer
        Normal  LoadBalancerManualChange  38s   g-cloudprovider     Firewall change required by security admin: `gcloud compute firewall-rules create k8s-fw-a5357045712f54237bf244180d442e8b --network installer-shared-vpc --description "{\"kubernetes.io/service-name\":\"openshift-ingress/router-default\", \"kubernetes.io/service-ip\":\"10.0.32.26\"}" --allow tcp:443,tcp:80 --source-ranges 0.0.0.0/0 --target-tags ci-op-jdhzhicc-f9971-wtgrt-master,ci-op-jdhzhicc-f9971-wtgrt-worker --project openshift-qe-shared-vpc`
        Normal  LoadBalancerManualChange  37s   g-cloudprovider     Firewall change required by security admin: `gcloud compute firewall-rules create a5357045712f54237bf244180d442e8b-hc --network installer-shared-vpc --description "" --allow tcp:30265 --source-ranges 130.211.0.0/22,209.85.152.0/22,209.85.204.0/22,35.191.0.0/16 --target-tags ci-op-jdhzhicc-f9971-wtgrt-master,ci-op-jdhzhicc-f9971-wtgrt-worker --project openshift-qe-shared-vpc`
        Normal  EnsuredLoadBalancer       36s   service-controller  Ensured load balancer
      

       

            rh-ee-nbrubake Nolan Brubaker
            rhn-support-hongli Hongan Li
            Zhaohua Sun Zhaohua Sun
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: