Uploaded image for project: 'Network Observability'
  1. Network Observability
  2. NETOBSERV-926

Connection tracking + Kafka might generate incorrect counters

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • NetObserv - Sprint 234, NetObserv - Sprint 235, NetObserv - Sprint 236, NetObserv - Sprint 237, NetObserv - Sprint 238, NetObserv - Sprint 239, NetObserv - Sprint 240, NetObserv - Sprint 241, NetObserv - Sprint 242
    • None
    • None
    • Previously, the when sending to kafka, the kafka message key was ignored causing error with connection tracking. Now the key is used for partitioning so each flow from the same connection are sent to the same processor.

      Connection tracking is processed per node (FLP) and maintains counter states. As traffic comes from EBPF agents that are also bound to a node, there is an automatic consistency in the n-tuple ids that are processed.

      When Kafka is used, FLP receives "random" flow data - loosing that n-tuple ids consistency. So the counter that it produces will be flawed, ie. they might not reflect the actual total of bytes/packets received.

        1. Screenshot 2023-06-20 at 4.07.21 PM.png
          337 kB
          Amogh Rameshappa Devapura
        2. Screenshot 2023-06-20 at 4.08.35 PM.png
          472 kB
          Amogh Rameshappa Devapura
        3. Screenshot 2023-06-20 at 4.09.06 PM.png
          377 kB
          Amogh Rameshappa Devapura
        4. Screenshot 2023-06-20 at 4.10.53 PM.png
          620 kB
          Amogh Rameshappa Devapura
        5. export-2023-06-22-15-54.csv
          8 kB
          Julien Pinsonneau

              ocazade@redhat.com Olivier Cazade
              jtakvori Joel Takvorian
              None
              None
              None
              Amogh Rameshappa Devapura Amogh Rameshappa Devapura
              None
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: