Uploaded image for project: 'Network Observability'
  1. Network Observability
  2. NETOBSERV-1275

Flows dedup broken for metrics in single-node case

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Undefined Undefined
    • netobserv-1.4
    • netobserv-1.4-candidate
    • Console Plugin
    • None
    • False
    • None
    • False
    • NetObserv - Sprint 241

      (Initially described in comments for NETOBSERV-1109)
      For traffic where src and dst are on the same node, the computed metrics are often wrong.
      To reproduce:

      • Deploy default netobserv / set sampling = 1
      • run "oc get pods -owide" to get the IP of the console plugin pod, and to get the name of the FLP pod that runs on the same node than the console plugin
        E.g:
      $ oc get pods -owide
      NAME                                            READY   STATUS    RESTARTS   AGE    IP             NODE                                         NOMINATED NODE   READINESS GATES
      flowlogs-pipeline-48mds                         1/1     Running   0          117m   10.129.0.78    ip-10-0-188-74.eu-west-3.compute.internal    <none>           <none>
      flowlogs-pipeline-4zzdf                         1/1     Running   0          117m   10.129.2.17    ip-10-0-180-132.eu-west-3.compute.internal   <none>           <none>
      flowlogs-pipeline-82zw4                         1/1     Running   0          117m   10.131.0.24    ip-10-0-145-95.eu-west-3.compute.internal    <none>           <none>
      flowlogs-pipeline-f8r89                         1/1     Running   0          117m   10.128.2.33    ip-10-0-193-116.eu-west-3.compute.internal   <none>           <none>
      flowlogs-pipeline-qbmjm                         1/1     Running   0          118m   10.128.0.54    ip-10-0-205-251.eu-west-3.compute.internal   <none>           <none>
      flowlogs-pipeline-zp99j                         1/1     Running   0          118m   10.130.0.146   ip-10-0-149-244.eu-west-3.compute.internal   <none>           <none>
      loki                                            1/1     Running   0          148m   10.131.0.21    ip-10-0-145-95.eu-west-3.compute.internal    <none>           <none>
      netobserv-controller-manager-7569476d6b-mfvhc   2/2     Running   0          56s    10.129.2.18    ip-10-0-180-132.eu-west-3.compute.internal   <none>           <none>
      netobserv-plugin-79765448f5-gfhq8               1/1     Running   0          27s    10.129.2.19    ip-10-0-180-132.eu-west-3.compute.internal   <none>           <none>
      
      

      => take plugin's IP 10.129.2.19 and FLP pod flowlogs-pipeline-4zzdf 

      • Run a couple of times "oc exec -it flowlogs-pipeline-4zzdf – curl https://10.129.2.19:9001 -k"
      • Open the console plugin in the browser, with filters: Source Name="flowlogs-pipeline-4zzdf", Destination Name=plugin . Keep "Show duplicates" box unchecked and "One way".
      • You should see one flow for every curl you did. When you display the "Direction" column, you should see that some are Ingress and others are Egress, randomly (if you're unlucky and only see Ingresses, run more curls).
      • Note the total bytes summary in the bottom bar
      • Switch to the Topology view, set slider to Resource scope
      • In advanced / display options, set Edge labels to Total bytes
      •  

      Expectation: The value on the edge label should more or less match the total bytes summary that you noted on the Flows table (with some small error margin due to rate computation)

      Actual result: The edge label value is much lower (like twice lower) than the value shown on Flows table. In fact, it accounts only for the Ingress flows and not the Egress ones displayed in that table.

        1. Capture d’écran du 2023-09-12 18-04-58.png
          Capture d’écran du 2023-09-12 18-04-58.png
          16 kB
        2. connection_tracking_table.png
          connection_tracking_table.png
          517 kB
        3. connection_tracking_topo.png
          connection_tracking_topo.png
          307 kB
        4. direction_filter.png
          direction_filter.png
          33 kB
        5. KAFKA_table.png
          KAFKA_table.png
          348 kB
        6. KAFKA_topo.png
          KAFKA_topo.png
          315 kB

            jtakvori Joel Takvorian
            jtakvori Joel Takvorian
            Amogh Rameshappa Devapura Amogh Rameshappa Devapura
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: