OVNKubernetes network Error getting pod annotations in a 400 memebers mesh

When we test OSSM with 400 members work load, the ovnkube pod ovnkube container shows following error and OSSM test cases failed randomly.

W0721 15:48:59.176665 2570 cni.go:89] Error getting pod annotations: Get "https://api-int.yuaxu-maistra-ovn.devcluster.openshift.com:6443/api/v1/namespaces/openshift-controller-manager/pods/controller-manager-v7fpb": unexpected EOF

Build Info:
OCP version: 4.6.39
network: OVNKubernetes
ovn rpm version:
ovn2.13-20.12.0-24.el8fdp.x86_64
ovn2.13-vtep-20.12.0-24.el8fdp.x86_64
ovn2.13-host-20.12.0-24.el8fdp.x86_64
ovn2.13-central-20.12.0-24.el8fdp.x86_64

OSSM operator version: 2.0.6.2
SMCP version: 1.1.16.2

How to reproduce:
1. Create an OCP cluster with OVNKubernetes network type in install-config.yaml
2. Deploy OSSM 2.0.6.2 operator and create SMCP 1.1.16.2
3. Create 400 namespaces and add them into a SMCP SMMR
4. Perform regular istio tasks such as traffic routing or request timeout and see random failures.