How has this gone unreported for 5+y? Or maybe it was injected recently as a result of the installer's CAPZ transition jhixson_redhat?

Anyway...

When deprovisioning an Azure cluster wherein a non-default baseDomainResourceGroup was specified, that baseDomainResourceGroup must be passed into the destroyer via the metadata object. Otherwise, the installer code assumes the base domain's DNS records should exist in the default resource group (something like <infraID>-rg) and when they're not found there, assumes they were already deleted, so this is not treated as a failure.

...The end result being leaked DNS records, and subsequent inability to create another cluster of the same name without scrubbing said records.