Uploaded image for project: 'Network Observability'
  1. Network Observability
  2. NETOBSERV-497

Remove aggregation stage in operator-defined pipeline

    • Icon: Story Story
    • Resolution: Done
    • Icon: Undefined Undefined
    • None
    • None
    • FLP, Operator, Prometheus
    • NetObserv - Sprint 222, NetObserv - Sprint 223, NetObserv - Sprint 224

      FLP can expose metrics to prometheus without necessarily needing an aggregation stage. The prometheus client provides natively some aggregations:

      • by defining dimensions (labels), data comes aggregated by these labels; so the metrics cardinality is unchanged, compared with a prior Aggregation stage
      • data is "aggregated" in time between each scrape intervals. Counters add up, Histograms generate stats (percentiles/count/sum), only Gauges will loose in accuracy since only their value at scrape time are being recorded.
      • note that, the "top-k" functionality of the Aggregation stage cannot be reproduced in the same way. Top-k can be performed at query-time with promQL, but an advantage of the pre-processed top-k offered by the aggregation stage is to reduce the amount of stored metrics, hence allowing a mitigation for high-cardinality labels.
      • Some aggregations functions, like "count number of services", can be obtained via promql instead of being a metric by itself.

      It seems like the metrics we're interested in for NetObserv operator don't need that aggregation stage most of the time, so we could remove it, thus optimizing FLP's pipeline

      See also PoC https://github.com/netobserv/flowlogs-pipeline/pull/266

      Note that, as said in the PoC, at the moment we must make a choice between having every metric going through aggregation stage, or having none. There's no in-between.

      There is a follow-up task to make FLP able to mix metrics with and without pre-aggregation: NETOBSERV-498

              jtakvori Joel Takvorian
              jtakvori Joel Takvorian
              Amogh Rameshappa Devapura Amogh Rameshappa Devapura
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: