When creating a HostedCluster with an invalid OIDC configuration (and the annotation to cleanup cloud resources) (For example, specifying a service signing key, but not prepopulating an S3 bucket with discovery docs), if the HostedCluster is deleted, the HostedCluster never goes away because the hosted control plane operator tries to delete the LoadBalancer services inside the guest cluster but cannot:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning SyncLoadBalancerFailed 3m41s (x1972 over 15h) service-controller (combined from similar events): Error syncing load balancer: failed to check if load balancer exists before cleanup: WebIdentityErr: failed to retrieve credentials
caused by: InvalidIdentityToken: Couldn't retrieve verification key from your identity provider, please reference AssumeRoleWithWebIdentity documentation for requirements
status code: 400, request id: b4f24c6b-0052-49de-b1d0-befcc1991081
We need to at the very least time out these deletions so that we can proceed with the cluster deletion.