Loading...

XML

Word

Printable

Details

Type: Bug
Resolution: Done
Priority: Normal
Fix Version/s: Logging 5.5.3
Affects Version/s: Logging 5.5.0
Component/s: Log Collection
Labels:
- devel_ack+
- rn-done-resolved

Story Points:
1
Blocked:
False
Blocked Reason:
None
Ready:
False
Docs QE Status:
NEW
QE Status:
VERIFIED
Release Note Text:

Hide
The collector configuration excludes collecting logs from collector,default-log-store, and viaualization pods. But if logs of these components are archived ina .gz file, the collector could not exclude these logs.

After this fix, any archived logs (*.gz) of collector, default-log-store, and visualization pods are also excluded.

Show
The collector configuration excludes collecting logs from collector,default-log-store, and viaualization pods. But if logs of these components are archived ina .gz file, the collector could not exclude these logs. After this fix, any archived logs (*.gz) of collector, default-log-store, and visualization pods are also excluded.

Sprint:
Log Collection - Sprint 222, Log Collection - Sprint 223, Log Collection - Sprint 224, Log Collection - Sprint 225

SFDC Cases Links:
SFDC Cases Counter:

Description

Observed the following log entry (among many similar entries) sent to a log store. This is a log from collector container.

{
  "hostname": "vimal-008-h9xm5-master-0.c.openshift-gce-devel.internal",
  "file": "/var/log/pods/openshift-logging_collector-ljgxr_aef8d573-5047-4e75-ba30-cf9dacc2db26/collector/0.log.20220720-114021.gz",
  "log_type": "infrastructure",
  "kubernetes": {
    "container_id": "cri-o://d8890861aa6478973d56c1513cb1bb87221d84fc1599424bcf55fee38120704d",
    "namespace_name": "openshift-logging",
    "namespace_labels": {
      "openshift.io/cluster-monitoring": "true",
      "olm.operatorgroup.uid/700fc211-df3d-4a21-b5b6-62fa1da7e014": "",
      "kubernetes.io/metadata.name": "openshift-logging",
      "olm.operatorgroup.uid/4bba7ff4-b796-4480-90f1-1bc9d3b55799": ""
    },
    "container_image": "image-registry.openshift-image-registry.svc:5000/openshift/vector:0.21-rh-gcp",
    "pod_name": "collector-ljgxr",
    "container_name": "collector",
    "pod_owner": "DaemonSet/collector",
    "pod_ip": "10.130.0.87",
    "labels": {
      "controller-revision-hash": "84f4bfff49",
      "logging-infra": "collector",
      "provider": "openshift",
      "pod-template-generation": "2",
      "component": "collector"
    },
    "pod_id": "aef8d573-5047-4e75-ba30-cf9dacc2db26",
    "annotations": {
      "scheduler.alpha.kubernetes.io/critical-pod": "",
      "openshift.io/scc": "log-collector-scc",
      "k8s.v1.cni.cncf.io/networks-status": "[{\n    \"name\": \"openshift-sdn\",\n    \"interface\": \"eth0\",\n    \"ips\": [\n        \"10.130.0.87\"\n    ],\n    \"default\": true,\n    \"dns\": {}\n}]",
      "k8s.v1.cni.cncf.io/network-status": "[{\n    \"name\": \"openshift-sdn\",\n    \"interface\": \"eth0\",\n    \"ips\": [\n        \"10.130.0.87\"\n    ],\n    \"default\": true,\n    \"dns\": {}\n}]"
    }
  },
  "level": "debug",
  "@timestamp": "2022-07-20T11:40:09.676561321Z",
  "message": "2022-07-20T11:40:09.676550Z DEBUG hyper::proto::h1::io: flushed 16384 bytes"
}

observing the field 'file', the logs are being read from a .gz file. This file can be present if there are files that have been rotated out far enough that it's been compressed.

The "exclude_paths_glob_patterns" currently only excludes '*.log', so need to change the pattern to exclude all files.

Alternate way to exclude container logs from reading is to use special labels defined by vector

https://vector.dev/docs/reference/configuration/sources/kubernetes_logs/#pod-exclusion