Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-9449

Failure to remove finalizer because of externalIPs being forbidden

    XMLWordPrintable

Details

    • ?
    • Moderate
    • Unspecified
    • If docs needed, set a value

    Description

      Description of problem:

      The CI caught[1] an issue that prevents a Service from having its finalizer removed and consequently having the Service deleted because the externalIP field is forbidden.

      2022-08-04T15:21:30.705681652Z I0804 15:21:30.705619       1 loadbalancer.go:3200] "Deleted load balancer" lbID="b87cba55-87d6-4f78-9cfc-3cb360916b31" service="e2e-test-ns-global-4vgqz/svc-with-ext-ippgd5d"
      2022-08-04T15:21:30.718837832Z E0804 15:21:30.718789       1 controller.go:310] error processing service e2e-test-ns-global-4vgqz/svc-with-ext-ippgd5d (will retry): failed to remove load balancer cleanup finalizer: services "svc-with-ext-ippgd5d" is forbidden: spec.externalIPs: Forbidden: externalIPs have been disabled
      2022-08-04T15:21:30.719052428Z I0804 15:21:30.719007       1 event.go:294] "Event occurred" object="e2e-test-ns-global-4vgqz/svc-with-ext-ippgd5d" fieldPath="" kind="Service" apiVersion="v1" type="Warning" reason="SyncLoadBalancerFailed" message="Error syncing load balancer: failed to remove load balancer cleanup finalizer: services \"svc-with-ext-ippgd5d\" is forbidden: spec.externalIPs: Forbidden: externalIPs have been disabled"
      2022-08-04T15:21:30.720797820Z E0804 15:21:30.720665       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"svc-with-ext-ippgd5d.17082d800f1ec40b", GenerateName:"", Namespace:"e2e-test-ns-global-4vgqz", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Service", Namespace:"e2e-test-ns-global-4vgqz", Name:"svc-with-ext-ippgd5d", UID:"ffbbf2f9-a423-4cdb-8f73-d3f869ade752", APIVersion:"v1", ResourceVersion:"93773", FieldPath:""}, Reason:"SyncLoadBalancerFailed", Message:"Error syncing load balancer: failed to remove load balancer cleanup finalizer: services \"svc-with-ext-ippgd5d\" is forbidden: spec.externalIPs: Forbidden: externalIPs have been disabled", Source:v1.EventSource{Component:"service-controller", Host:""}, FirstTimestamp:time.Date(2022, time.August, 4, 15, 21, 30, 718766091, time.Local), LastTimestamp:time.Date(2022, time.August, 4, 15, 21, 30, 718766091, time.Local), Count:1, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "svc-with-ext-ippgd5d.17082d800f1ec40b" is forbidden: unable to create new content in namespace e2e-test-ns-global-4vgqz because it is being terminated' (will not retry!)
      

      The network configuration:

      apiVersion: config.openshift.io/v1
      items:

      • apiVersion: config.openshift.io/v1
        kind: Network
        metadata:
        creationTimestamp: "2022-08-04T13:40:05Z"
        generation: 4
        manager: cluster-bootstrap
        operation: Update
        time: "2022-08-04T13:40:05Z"
        name: cluster
        resourceVersion: "89402"
        uid: 28b1c778-0856-4ed6-8d30-95953a8c59a3
        spec:
        clusterNetwork:
      • cidr: 10.128.0.0/14
        hostPrefix: 23
        externalIP:
        policy: {}
        networkType: OpenShiftSDN
        serviceNetwork:
      • 172.30.0.0/16
        status:
        clusterNetwork:
      • cidr: 10.128.0.0/14
        hostPrefix: 23
        clusterNetworkMTU: 1400
        networkType: OpenShiftSDN
        serviceNetwork:
      • 172.30.0.0/16

      Seems to be similar to https://access.redhat.com/solutions/5560601

      [1] https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-shiftstack-shiftstack-ci-main-periodic-4.12-e2e-openstack-serial/1555179360489574400

      Version-Release number of selected component (if applicable):

      How reproducible:

      Steps to Reproduce:
      1.
      2.
      3.

      Actual results:

      Expected results:

      Additional info:

      Attachments

        Activity

          People

            Unassigned Unassigned
            mdemaced Maysa De Macedo Souza
            Jon Uriarte Jon Uriarte
            Red Hat Employee
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: