-
Bug
-
Resolution: Done
-
Blocker
-
Logging 5.4.0
-
False
-
None
-
False
-
NEW
-
OBSDA-108 - Distribute an alternate Vector Log Collector
-
VERIFIED
-
-
Logging (Core) - Sprint 218, Logging (Core) - Sprint 219, Logging (Core) - Sprint 220
Version:
Vector TP
Description of the problem:
The log records from Vector has missing prefix '@' for timestamp field.
{
"kubernetes": {
"container_name": "log-generator",
"flat_labels": [
"component=test"
],
"pod_uid": "116bbaeb-1d84-4316-9af3-9f370cafad6d",
"pod_ip": "10.217.0.150",
"container_id": "cri-o://eb812e2a8bdeeb1f86af3bfc1ec6484ed9f779946b24f5c9507562c797e5d205",
"container_image": "centos:centos7",
"namespace_name": "app1",
"pod_name": "json-log-generator",
"pod_node_name": "crc-hsl9k-master-0"
},
"log_type": "application",
"write-index": "app-write",
"level": "info",
"message": "{\"index\": 0, \"facility_key\": \"local0\", \"msgcontent\": \"My life is my message\", \"timestamp\": \"2022-03-02 15:43:01\", \"severity_key\": \"Informational\", \"jndex\": 999, \"mykey\": \"myvalue\"}",
"timestamp": "2022-03-02T15:43:01.832044218Z"
}
The prefix is needed for Vector log record to be backwards compatible with the data model of Fluentd. For adding the index filter in Kibana, Kibana expects the time filter timestamp with the prefix '@' . Also the prefix denotes a field that is reserved for a particular use. By default, most tools look for “@timestamp” . https://docs.openshift.com/container-platform/4.9/logging/cluster-logging-exported-fields.html#timestamp.
Example Fluentd record for reference.
{
"_index": "app-000001",
"_type": "_doc",
"_id": "NzBhNzFkY2ItOTE4Zi00Njc1LTg3OWItMmJkMzljMDJkOTJj",
"_version": 1,
"_score": null,
"_source": {
"kubernetes": {
"container_image_id": "quay.io/openshifttest/ocp-logtest@sha256:16232868ba1143721b786dbabb3f7384645acb663fadb4af48e9ea1228a67635",
"container_name": "logging-centos-logtest",
"namespace_id": "b348298d-4b37-4760-8141-c98d42e9df58",
"flat_labels": [
"run=centos-logtest-qa",
"test=centos-logtest-qa"
],
"pod_ip": "10.128.2.15",
"host": "ikanse-11-n9h24-worker-a-7cm58.c.openshift-qe.internal",
"master_url": "https://kubernetes.default.svc",
"pod_id": "f69cf8d5-9b41-4fbd-abb1-844690630e24",
"namespace_labels": {
"kubernetes_io/metadata_name": "test"
},
"container_image": "quay.io/openshifttest/ocp-logtest@sha256:16232868ba1143721b786dbabb3f7384645acb663fadb4af48e9ea1228a67635",
"namespace_name": "test",
"pod_name": "logging-centos-logtest-qa-s79k7"
},
"viaq_msg_id": "NzBhNzFkY2ItOTE4Zi00Njc1LTg3OWItMmJkMzljMDJkOTJj",
"level": "unknown",
"message": "{\"message\": \"MERGE_JSON_LOG=true\", \"level\": \"debug\",\"Layer1\": \"layer1 0\", \"layer2\": {\"name\":\"Layer2 1\", \"tips\":\"Decide by PRESERVE_JSON_LOG\"}, \"StringNumber\":\"10\", \"Number\": 10,\"foo.bar\":\"Dot Item\",\"{foobar}\":\"Brace Item\",\"[foobar]\":\"Bracket Item\", \"foo:bar\":\"Colon Item\",\"foo bar\":\"Space Item\" }",
"docker": {
"container_id": "8df7a9f852cf98538c268895b673fb82a5ff4c1f1a028d73c49d6afb754e9439"
},
"hostname": "ikanse-11-n9h24-worker-a-7cm58.c.openshift-qe.internal",
"log_type": "application",
"@timestamp": "2022-03-09T04:17:43.727230+00:00",
"pipeline_metadata": {
"collector": {
"received_at": "2022-03-09T04:17:43.728883+00:00",
"name": "fluentd",
"inputname": "fluent-plugin-systemd",
"version": "1.7.4 1.6.0",
"ipaddr4": "10.0.128.3"
}
}
},
"fields": {
"@timestamp": [
"2022-03-09T04:17:43.727Z"
],
"pipeline_metadata.collector.received_at": [
"2022-03-09T04:17:43.728Z"
]
},
"sort": [
1646799463727
]
}
