Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-4518

SNO DU Scale tests with telco workload show high cpu usage (~1.3 cores) from collector pod

XMLWordPrintable

    • False
    • None
    • False
    • NEW
    • NEW
    • Proposed

      Description of problem:

      Collector pod deployed by the cluster-logging operator is using 1.3 cores of cpu on SNO DUs

      Version-Release number of selected component (if applicable):

      cluster-logging.v5.7.5
      
      Tested on:
      4.13.4
      4.14.0-ec.4
      4.14.0-0.ci-2023-09-05-120503
      
      

      How reproducible:

      1. Configure SNO with DU profile. 
      2. Deploy cluster-logging operator and configure clusterLogging.yaml and clusterLogForwarding.yaml
      3. Deploy Telco workloads and gather resource usage metrics
      

      Actual results:

      CPU Usage in Cores:

      Iteration 4.13.4 4.14.0_ec.4 4.14.0-0.ci-2023-09-05-120503
      1 0.7119231194 0.7413879821 1.314202569
      2 0.6751107603 0.7218595507 1.319234136
      3 0.6984978924 0.7775478173 1.306324442
      4 0.7541069158 0.7327866483 1.329228295
      5 0.7484353611 0.7873075163 1.300583959
      6 0.7753512426 0.8063833829 1.299729106
      Avg 0.7272375486 0.76 1.31
      CPU usage is high on all three releases. There is also an increase of ~0.58 cores between 4.14.0-0.ci-2023-09-05-120503 and 4.13.4. 

      Expected results:

       

      Additional info:

      clusterLogging.yaml
      ---
      apiVersion: logging.openshift.io/v1
      kind: ClusterLogging
      metadata:
        name: instance
        namespace: openshift-logging
      spec:
        managementState: "Managed"
        curation:
          type: "curator"
          curator:
            schedule: "30 3 * * *"
        collection:
          logs:
            type: "vector"
            vector: {}
      
      clusterLogForwarder.yaml
      ---
      apiVersion: "logging.openshift.io/v1"
      kind: ClusterLogForwarder
      metadata:
        name: instance
        namespace: openshift-logging
      spec:
        outputs:
          - type: "kafka"
            name: kafka-open
            url: tcp://e28-h29-r750.alias.bos.scalelab.redhat.com:9092/e28-h37-r750
        pipelines:
          - name: audit-logs
            inputRefs:
             - audit
            outputRefs:
             - kafka-open
          - name: infrastructure-logs
            inputRefs:
             - infrastructure
            outputRefs:
             - kafka-open
       

        1. collector-log.tar.gz
          5.85 MB
          Noreen Chhabra
        2. collector-pod-describe.log
          8 kB
          Noreen Chhabra

              Unassigned Unassigned
              nchhabra@redhat.com Noreen Chhabra
              Anping Li Anping Li
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: