[1996660] [4.8] Goroutine count and memory remains high after VMIs are removed

This bug was initially created as a copy of Bug #1996658

I am copying this bug to track the fix for 4.8 as well.

Description of problem:
Originally tracked in https://github.com/kubevirt/kubevirt/issues/6056
When creating VMIs and deleting them, virt-handler does not clean up goroutines properly.

How reproducible:
Create n VMIs, wait until they are all running, delete n VMIs, wait until they are all deleted, sleep 5s, and repeat. Scenarios for 10, 20, 40, 60, 80, 100, 200, 300 VMIs shows an increase in virt-handler mem/cpu usage and go routines, however only cpu usage returns back to the expected level when there are 0 VMIs.

Steps to Reproduce:
1. Create 100 VMIs (not needed to start them), can use Kubevirt Density Test
2. Delete 100 VMIs
3. Observe goroutines metric on virt-handler

Actual results:
Goroutines metric does not go down all the way after deletion.

Expected results:
Goroutines metric should go down all the way after deletion.

Fixed in https://github.com/kubevirt/kubevirt/pull/6176
Backport for 4.8 is pending until this cherry-pick merges: https://github.com/kubevirt/kubevirt/pull/6227