Loading...

XML

Word

Printable

Details

Type: Bug
Resolution: Done-Errata
Priority: Normal
Fix Version/s: Logging 5.7.4
Affects Version/s: Logging 5.5.0
Component/s: Log Collection
Labels:
- devel_ack+

Blocked:
False
Blocked Reason:
None
Ready:
False
Docs QE Status:
NEW
QE Status:
VERIFIED
Release Note Text:

Hide
Before this fix, when forwarding logs to CloudWatch, namespaceUUID was not appended to the logGroupName. Now with the fix, namespaceUUID is appended, so a logGroupName in CloudWatch appears as
"logGroupName": "vectorcw.b443fb9e-bd4c-4b6a-b9d3-c0097f9ed286".

Show
Before this fix, when forwarding logs to CloudWatch, namespaceUUID was not appended to the logGroupName. Now with the fix, namespaceUUID is appended, so a logGroupName in CloudWatch appears as "logGroupName": "vectorcw.b443fb9e-bd4c-4b6a-b9d3-c0097f9ed286".
Release Note Type:
Bug Fix

Sprint:
Log Collection - Sprint 235, Log Collection - Sprint 236, Log Collection - Sprint 239

SFDC Cases Links:
SFDC Cases Counter:

Description

An issue for this is already opened in upstream by cahartma@redhat.com https://github.com/vectordotdev/vector/issues/12824 and also mentioned in user story ~~LOG-2223~~

Created the issue to track the progress on bug fix.

Version of components:

clusterlogging.v5.5.0

elasticsearch-operator.v5.5.0

Server Version: 4.10.0-0.nightly-2022-06-07-181847

Kubernetes Version: v1.23.5+3afdacb

Description of the issue:

When forwarding logs to Cloudwatch using Vector as collector and specifying groupBy: namespaceUUID in the ClusterLogForwarder, the namespaceUUID is not added to the logGroupName in Cloudwatch.

Steps to reproduce the issue:

1 1 Create secret for forwarding to Cloudwatch.

export ACCESS_KEY_ID=$(oc get secret aws-creds -n kube-system -o json | jq -r '.data.aws_access_key_id'|base64 -d)

export SECRET_ACCESS_KEY=$(oc get secret  aws-creds -n kube-system -o json |jq -r '.data.aws_secret_access_key'|base64 -d)

oc -n openshift-logging create secret generic cw-secret \
--from-literal=aws_access_key_id="${ACCESS_KEY_ID}" \
--from-literal=aws_secret_access_key="${SECRET_ACCESS_KEY}"

2 Deploy a log generator app.

oc new-project test

oc new-app https://raw.githubusercontent.com/openshift/verification-tests/master/testdata/logging/loggen/container_json_log_template.json

3 Create ClusterLogForwarder instance to forward logs to CloudWatch.

apiVersion: "logging.openshift.io/v1"
kind: ClusterLogForwarder
metadata:
  name: instance 
  namespace: openshift-logging 
spec:
  outputs:
   - name: cw
     type: cloudwatch
     cloudwatch:
       groupBy: namespaceUUID
       groupPrefix: vectorcw
       region: us-east-2
     secret:
        name: cw-secret
  pipelines:
    - name: infra-logs
      inputRefs:
        - infrastructure
        - audit
        - application
      outputRefs:
        - cw

4 Create ClusterLogging instance.

apiVersion: "logging.openshift.io/v1"
kind: "ClusterLogging"
metadata:
  name: "instance" 
  namespace: "openshift-logging"
spec:
  managementState: "Managed"  
  collection:
    logs:
      type: "vector"  
      vector: {}

5 Check the log group in Cloudwatch. The app logs is missing namespaceUUID.

aws logs describe-log-groups --log-group-name-prefix vectorcw --region=us-east-2

{
    "logGroups": [
        {
            "logGroupName": "vectorcw.",
            "creationTime": 1654668657739,
            "metricFilterCount": 0,
            "arn": "arn:aws:logs:us-east-2:301721915996:log-group:vectorcw.:*",
            "storedBytes": 0
        },
        {
            "logGroupName": "vectorcw.audit",
            "creationTime": 1654668566324,
            "metricFilterCount": 0,
            "arn": "arn:aws:logs:us-east-2:301721915996:log-group:vectorcw.audit:*",
            "storedBytes": 0
        },
        {
            "logGroupName": "vectorcw.infrastructure",
            "creationTime": 1654668566343,
            "metricFilterCount": 0,
            "arn": "arn:aws:logs:us-east-2:301721915996:log-group:vectorcw.infrastructure:*",
            "storedBytes": 0
        }
    ]
}