design unsupported escape hatch mechanism custom HS/KV vms configuration

CNV QE, field engineers, and developers often need to test hypershift kubevirt in a way that isn't officially supported yet, and this often involves needing to modify the kubevirt VM's spec to enable some sort of feature, add an interface/volume, or something else along those lines.

We need to design a mechanism that works as an escape hatch to allow these sorts of unsupported modifications to be experimented with easily. This mechanism should not be a part of the official Hypershift APIs, but instead something that people can influence via an annotation or similar means.

It's likely this feature will serve as a way for us to grant temporary support exceptions to customers as well. 

This can be achieved using an annotation with a json patch in it. Below is an example of how such a json patch might be placed on a NodePool to influence the VMs generated by the NodePool to have a secondary interface. 

apiVersion: hypershift.openshift.io/v1beta1
kind: NodePool
metadata:
  annotations:
    hypershift.openshift.io/kubevirt-vm-jsonpatch: |-
      [
        {
          "op": "add",
          "path": "/spec/template/spec/networks",
          "value": {"name": secondary, multus: networkName: mynetwork}
        },
        {
          "op": "add",
          "path": "/spec/template/spec/domain/devices/interfaces",
          "value": {"name": secondary, bridge: {}}
        }
      ]