Bug 1859004 - Sometimes the eventrouter couldn't gather event logs.

Description of problem:
Deploy logging and eventrouter, the eventrouter can't gather event logs, lots of error message in the eventrouter pod:

I0721 00:23:52.890323 1 reflector.go:240] Listing and watching *v1.Event from github.com/openshift/eventrouter/vendor/k8s.io/client-go/informers/factory.go:73
E0721 00:23:52.904347 1 reflector.go:205] github.com/openshift/eventrouter/vendor/k8s.io/client-go/informers/factory.go:73: Failed to list *v1.Event: v1.EventList: Items: []v1.Event: v1.Event: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, but found u, error found in #10 byte of ...|:{},"k:

{\"uid\":\"30|..., bigger context ...|}

,"f:metadata":{"f:ownerReferences":{".":{},"k:

{\"uid\":\"303e9c69-80bf-4001-9ccf-25c8f1f4c14e\"}

":{|...
I0721 00:23:53.904461 1 reflector.go:240] Listing and watching *v1.Event from github.com/openshift/eventrouter/vendor/k8s.io/client-go/informers/factory.go:73
E0721 00:23:53.922332 1 reflector.go:205] github.com/openshift/eventrouter/vendor/k8s.io/client-go/informers/factory.go:73: Failed to list *v1.Event: v1.EventList: Items: []v1.Event: v1.Event: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, but found u, error found in #10 byte of ...|:{},"k:

{\"uid\":\"30|..., bigger context ...|}

,"f:metadata":{"f:ownerReferences":{".":{},"k:

{\"uid\":\"303e9c69-80bf-4001-9ccf-25c8f1f4c14e\"}

":{|...

Version-Release number of selected component (if applicable):
ose-logging-eventrouter-v4.5.0-202007172106.p0
cluster version: 4.5.0-0.nightly-2020-07-20-152128

How reproducible:
In some clusters, it's 100% reproducible, in some clusters, no such issue

Steps to Reproduce:
1.deploy logging
2.deploy eventrouter in openshift-logging namespace with:
kind: Template
apiVersion: v1
metadata:
name: eventrouter-template
annotations:
description: "A pod forwarding kubernetes events to cluster logging stack."
tags: "events,EFK,logging, cluster-logging"
objects:

• kind: ServiceAccount
  apiVersion: v1
  metadata:
  name: cluster-logging-eventrouter
  namespace: ${NAMESPACE}
• kind: ClusterRole
  apiVersion: v1
  metadata:
  name: event-reader
  rules:
• apiGroups: [""]
  resources: ["events"]
  verbs: ["get", "watch", "list"]
• kind: ClusterRoleBinding
  apiVersion: v1
  metadata:
  name: event-reader-binding
  subjects:
• kind: ServiceAccount
  name: cluster-logging-eventrouter
  namespace: ${NAMESPACE}
  roleRef:
  kind: ClusterRole
  name: event-reader
• kind: ConfigMap
  apiVersion: v1
  metadata:
  name: cluster-logging-eventrouter
  namespace: ${NAMESPACE}
  data:
  config.json: |- { "sink": "stdout" }
• kind: Deployment
  apiVersion: apps/v1
  metadata:
  name: cluster-logging-eventrouter
  namespace: ${NAMESPACE}
  labels:
  component: eventrouter
  logging-infra: eventrouter
  provider: openshift
  spec:
  selector:
  matchLabels:
  component: eventrouter
  logging-infra: eventrouter
  provider: openshift
  replicas: 1
  template:
  metadata:
  labels:
  component: eventrouter
  logging-infra: eventrouter
  provider: openshift
  name: cluster-logging-eventrouter
  spec:
  serviceAccount: cluster-logging-eventrouter
  containers:
• name: kube-eventrouter
  image: ${IMAGE}
  imagePullPolicy: IfNotPresent
  resources:
  limits:
  memory: ${MEMORY}
  requests:
  cpu: ${CPU}
  memory: ${MEMORY}
  volumeMounts:
• name: config-volume
  mountPath: /etc/eventrouter
  volumes:
• name: config-volume
  configMap:
  name: cluster-logging-eventrouter
  parameters:
• name: IMAGE
  displayName: Image
  value: "image-registry.openshift-image-registry.svc:5000/openshift/ose-logging-eventrouter:latest"
• name: MEMORY
  displayName: Memory
  value: "128Mi"
• name: CPU
  displayName: CPU
  value: "100m"
• name: NAMESPACE
  displayName: Namespace
  value: "openshift-logging"
  3.check eventrouter pod logs

Actual results:

Expected results:

Additional info:
must-gather: http://file.apac.redhat.com/~qitang/must-gather.tar.gz