Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-1230

Bug 1859004 - Sometimes the eventrouter couldn't gather event logs.

    XMLWordPrintable

Details

    • Logging (Core) - Sprint 199, Logging (Core) - Sprint 200
    • Passed
    • NEW
    • VERIFIED

    Description

      Description of problem:
      Deploy logging and eventrouter, the eventrouter can't gather event logs, lots of error message in the eventrouter pod:

      I0721 00:23:52.890323 1 reflector.go:240] Listing and watching *v1.Event from github.com/openshift/eventrouter/vendor/k8s.io/client-go/informers/factory.go:73
      E0721 00:23:52.904347 1 reflector.go:205] github.com/openshift/eventrouter/vendor/k8s.io/client-go/informers/factory.go:73: Failed to list *v1.Event: v1.EventList: Items: []v1.Event: v1.Event: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, but found u, error found in #10 byte of ...|:{},"k:

      {\"uid\":\"30|..., bigger context ...|}

      ,"f:metadata":{"f:ownerReferences":{".":{},"k:

      {\"uid\":\"303e9c69-80bf-4001-9ccf-25c8f1f4c14e\"}

      ":{|...
      I0721 00:23:53.904461 1 reflector.go:240] Listing and watching *v1.Event from github.com/openshift/eventrouter/vendor/k8s.io/client-go/informers/factory.go:73
      E0721 00:23:53.922332 1 reflector.go:205] github.com/openshift/eventrouter/vendor/k8s.io/client-go/informers/factory.go:73: Failed to list *v1.Event: v1.EventList: Items: []v1.Event: v1.Event: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, but found u, error found in #10 byte of ...|:{},"k:

      {\"uid\":\"30|..., bigger context ...|}

      ,"f:metadata":{"f:ownerReferences":{".":{},"k:

      {\"uid\":\"303e9c69-80bf-4001-9ccf-25c8f1f4c14e\"}

      ":{|...

      Version-Release number of selected component (if applicable):
      ose-logging-eventrouter-v4.5.0-202007172106.p0
      cluster version: 4.5.0-0.nightly-2020-07-20-152128

      How reproducible:
      In some clusters, it's 100% reproducible, in some clusters, no such issue

      Steps to Reproduce:
      1.deploy logging
      2.deploy eventrouter in openshift-logging namespace with:
      kind: Template
      apiVersion: v1
      metadata:
      name: eventrouter-template
      annotations:
      description: "A pod forwarding kubernetes events to cluster logging stack."
      tags: "events,EFK,logging, cluster-logging"
      objects:

      • kind: ServiceAccount
        apiVersion: v1
        metadata:
        name: cluster-logging-eventrouter
        namespace: ${NAMESPACE}
      • kind: ClusterRole
        apiVersion: v1
        metadata:
        name: event-reader
        rules:
      • apiGroups: [""]
        resources: ["events"]
        verbs: ["get", "watch", "list"]
      • kind: ClusterRoleBinding
        apiVersion: v1
        metadata:
        name: event-reader-binding
        subjects:
      • kind: ServiceAccount
        name: cluster-logging-eventrouter
        namespace: ${NAMESPACE}
        roleRef:
        kind: ClusterRole
        name: event-reader
      • kind: ConfigMap
        apiVersion: v1
        metadata:
        name: cluster-logging-eventrouter
        namespace: ${NAMESPACE}
        data:
        config.json: |- { "sink": "stdout" }
      • kind: Deployment
        apiVersion: apps/v1
        metadata:
        name: cluster-logging-eventrouter
        namespace: ${NAMESPACE}
        labels:
        component: eventrouter
        logging-infra: eventrouter
        provider: openshift
        spec:
        selector:
        matchLabels:
        component: eventrouter
        logging-infra: eventrouter
        provider: openshift
        replicas: 1
        template:
        metadata:
        labels:
        component: eventrouter
        logging-infra: eventrouter
        provider: openshift
        name: cluster-logging-eventrouter
        spec:
        serviceAccount: cluster-logging-eventrouter
        containers:
      • name: kube-eventrouter
        image: ${IMAGE}
        imagePullPolicy: IfNotPresent
        resources:
        limits:
        memory: ${MEMORY}
        requests:
        cpu: ${CPU}
        memory: ${MEMORY}
        volumeMounts:
      • name: config-volume
        mountPath: /etc/eventrouter
        volumes:
      • name: config-volume
        configMap:
        name: cluster-logging-eventrouter
        parameters:
      • name: IMAGE
        displayName: Image
        value: "image-registry.openshift-image-registry.svc:5000/openshift/ose-logging-eventrouter:latest"
      • name: MEMORY
        displayName: Memory
        value: "128Mi"
      • name: CPU
        displayName: CPU
        value: "100m"
      • name: NAMESPACE
        displayName: Namespace
        value: "openshift-logging"
        3.check eventrouter pod logs

      Actual results:

      Expected results:

      Additional info:
      must-gather: http://file.apac.redhat.com/~qitang/must-gather.tar.gz

      Attachments

        Issue Links

          Activity

            People

              vparfono Vitalii Parfonov
              jcantril@redhat.com Jeffrey Cantrill
              Qiaoling Tang Qiaoling Tang
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: