Uploaded image for project: 'Network Observability'
  1. Network Observability
  2. NETOBSERV-186

Research: configuration-defined pipelines

    • Icon: Story Story
    • Resolution: Done
    • Icon: Undefined Undefined
    • None
    • None
    • FLP
    • NetObserv - Sprint 215

      It could be interesting to define the pipeline stages and order via a configuration file (e.g. to be able to split our component in multiple sub-components running in different services, if there are scalability reasons for it). This task consists on:

      1. Investigate existing DSLs for defining data pipelines versus creating our own, simpler YAML format for defining the stages and the order. Another possibility is to adopt and extend the current flowlogs2metrics format. E.g.: https://github.com/netobserv/flowlogs2metrics/blob/main/network_definitions/bandwidth_per_src_subnet.yaml
      2. Investigate if is worth to define the different stages of the pipeline as Go modules, so we can load them dynamically when parsing the definition files from the previous bullet. We need to consider Go modules' limitations.

            mmaciasl@redhat.com Mario Macias (Inactive)
            mmaciasl@redhat.com Mario Macias (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: