Description of problem: Created AddOnDeploymentConfig and configure ManagedClusterAddOn observability-controller in the proxy environment. But metrics data still failed to forward data to the hub, the error is:

```
level=warn caller=logger.go:50 ts=2023-09-25T09:10:02.692400252Z component=forwarder component=metricsclient msg="error: failed to forward request happened at time: 563.752129ms"
level=warn caller=logger.go:50 ts=2023-09-25T09:10:08.257487112Z component=forwarder component=metricsclient msg="failed to forward request" err="Post \"https://observatorium-api-open-cluster-management-observability.apps.qe6-vmware-ibm.install.dev09.red-chesterfield.com/api/metrics/v1/default/api/v1/receive\": context deadline exceeded"
level=warn caller=logger.go:50 ts=2023-09-25T09:10:08.257557246Z component=forwarder component=metricsclient msg="error: failed to forward request happened at time: 1.400429871s"
level=warn caller=logger.go:50 ts=2023-09-25T09:10:11.263752279Z component=forwarder component=metricsclient msg="failed to forward request" err="Post \"https://observatorium-api-open-cluster-management-observability.apps.qe6-vmware-ibm.install.dev09.red-chesterfield.com/api/metrics/v1/default/api/v1/receive\": dial tcp: lookup observatorium-api-open-cluster-management-observability.apps.qe6-vmware-ibm.install.dev09.red-chesterfield.com on 172.30.0.10:53: server misbehaving"

```
 

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:

1. managed cluster are configured to connect hub with proxy
2. create AddOnDeploymentConfig with the proxy information
   ```
   apiVersion: addon.open-cluster-management.io/v1alpha1
   kind: AddOnDeploymentConfig
   metadata:
     name: obsaddondeployconfig
     namespace: test-cluster
   spec:
     proxyConfig:
       httpProxy:***(hide the data for security)
       noProxy: ".cluster.local,.svc"
   ```
3. add this AddOnDeploymentConfig into the ManagedClusterAddOn observability-controller
   ```

apiVersion: addon.open-cluster-management.io/v1alpha1
kind: ManagedClusterAddOn
metadata:
creationTimestamp: "2023-09-22T03:48:34Z"
generation: 2
labels:
owner: multicluster-observability-operator
name: observability-controller
namespace: test-cluster
ownerReferences:

• apiVersion: addon.open-cluster-management.io/v1alpha1
  blockOwnerDeletion: true
  controller: true
  kind: ClusterManagementAddOn
  name: observability-controller
  uid: 72eaf234-c21f-47ca-beb8-effb1057e924
  resourceVersion: "2165517"
  uid: abc8ab54-623d-451a-88e4-11673aefa905
  spec:
  configs:
• group: addon.open-cluster-management.io
  name: obsaddondeployconfig
  namespace: test-cluster
  resource: addondeploymentconfigs
  installNamespace: open-cluster-management-addon-observability
  ```

1. the metrics data still not forward to the hub

Actual results:

Expected results:

Additional info: