Uploaded image for project: 'Network Observability'
  1. Network Observability
  2. NETOBSERV-980

Flows dropped due to cert issue

    XMLWordPrintable

Details

    • False
    • None
    • False
    • NetObserv - Sprint 242, NetObserv - Sprint 243
    • Important

    Description

      I've observed flows being dropped due to the following cert issue:

      time=2023-04-12T17:41:25Z level=info component=client error=Post "https://lokistack-gateway-http.netobserv.svc.cluster.local:8080/api/logs/v1/network/loki/api/v1/push": unable to load specified CA cert /var/loki-certs-ca/service-ca.crt: open /var/loki-certs-ca/service-ca.crt: no such file or directory fields.level=warn fields.msg=error sending batch, will retry host=lokistack-gateway-http.netobserv.svc.cluster.local:8080 module=export/loki status=-1

      This periodically affects FLP pods and results in flows being dropped rather than being written to Loki. The problem does seem to self-correct but still is causing us to lose flow data. Attaching a screenshot of this behavior visualized from my most recent PerfScale test.

      Attachments

        Issue Links

          split from

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. NETOBSERV-902 QE: Run performance tests for 1.2 release

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. NETOBSERV-1068 Run performance tests for 1.3 release

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. NETOBSERV-1142 Run performance tests for 1.4 release

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. NETOBSERV-1250 Run long-lived cluster test with NetObserv 1.4

          • Normal - To be worked after urgent and high priorities are resolved. This term is chosen when the severity of an issue is relatively close to the level of effort to fix it. The existence of an easily implemented workaround can also lead to this priority level instead of a higher priority.
          • Closed
          links to

          GitHub openshift/openshift-docs#56244: Adds Network Observability 1.2 release notes

          GitHub openshift/openshift-docs#63934: Network Observability 1.4 Release Notes

          Activity

            People

              rhn-support-sarthoma Sara Thomas
              nweinber1 Nathan Weinberg
              Nathan Weinberg Nathan Weinberg
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: