Uploaded image for project: 'Network Observability'
  1. Network Observability
  2. NETOBSERV-973

Dropped flows seen due to connection tracking

    • False
    • None
    • False
    • NetObserv - Sprint 234
    • Critical

      When connection tracking is enabled, loki ingestion alert is observed due to drop in flows.
      FLP pods show error:

      time=2023-04-03T16:52:28Z level=info component=client error=server returned HTTP status 400 Bad Request (400): entry for stream '{DstK8S_Namespace="netobserv", DstK8S_OwnerName="lokistack-gateway", SrcK8S_Namespace="netobserv", SrcK8S_OwnerName="flowlogs-pipeline", _RecordType="newConnection", app="netobserv-flowcollector"}' has timestamp too old: 1677-09-21T00:12:43Z, oldest acceptable timestamp is: 2023-03-27T16:52:28Z fields.level=error fields.msg=final error sending batch host=lokistack-gateway-http.netobserv.svc.cluster.local:8080 module=export/loki status=40062time=2023-04-03T16:52:32Z level=debug msg=ticker signal: invoking action with 775 entries component=utils.Batcher 

      Observed on a 4.10, .13 cluster with the latest rc operator image. Using 1x.small size lokistack with 3 master and 3 worked with no load running

        1. Flows_to_Ended.png
          Flows_to_Ended.png
          360 kB
        2. End_to_flows_1043_to_1048.png
          End_to_flows_1043_to_1048.png
          339 kB
        3. Flows_to_All_1050.png
          Flows_to_All_1050.png
          331 kB
        4. Flows_to_Conv_1050.png
          Flows_to_Conv_1050.png
          337 kB

            jpinsonn@redhat.com Julien Pinsonneau
            rhn-support-aramesha Amogh Rameshappa Devapura
            Amogh Rameshappa Devapura Amogh Rameshappa Devapura
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: