-
Bug
-
Resolution: Obsolete
-
Normal
-
None
-
False
-
None
-
False
-
NEW
-
NEW
-
Bug Fix
-
-
Proposed title of this feature request
cx noticed log JSON change after migrating fluentd to vector. For instance: ・istiod.log before: ip-10-249-81-55.ap-northeast-1.compute.internal istiod-basic-7cd97fbd-89nvb 2024-05-16T05:24:35.873035Z info ads RDS: PUSH request for node:httpd-test-64f98c577d-qnlkv.hitachi-test resources:16 size:24.7kB cached:0/16 after: istiod-basic-7cd97fbd-89nvb 2024-05-16T05:24:35.873035Z info ads RDS: PUSH request for node:httpd-test-64f98c577d-qnlkv.hitachi-test resources:16 size:24.7kB cached:0/16 ⇒ip-10-249-81-55.ap-northeast-1.compute.internal is gone.
What is the nature and description of the request?
Workaround: made change to rsyslog.conf(cx's output server) as follows: property(name="$!kubernetes!host") ↓ property(name="$!hostname")
Reason for the WA can see host format is different: Fluentd JSON: { "_index": "app-000001", "_type": "_doc", "_id": "NTk2NWI1ODktZDk1My00MjcyLWE4ZmQtNThjYWRiZDVlYTAw", "_version": 1, "_score": null, "_source": { "kubernetes": { "container_image_id": "registry.redhat.io/openshift-service-mesh/pilot-rhel8@sha256:b451f504cb2078f363a4b184e4223289908ddf499f68bdee99f65350035276a6", "container_name": "discovery", "namespace_id": "1dcdcbf8-9f58-456a-9a6f-21e00fdbd669", "flat_labels": [ "app=istiod", "istio=istiod", "istio_io_rev=basic", "maistra-control-plane=istio-system", "pod-template-hash=c754cb9cf", "sidecar_istio_io_inject=false" ], "pod_ip": "10.131.2.31", "host": "hollytest-wbxlb-worker-eastus1-j8r6x", "master_url": "https://kubernetes.default.svc", "pod_id": "f051a874-95f1-4817-9bef-69ed48a46827", "namespace_labels": { "pod-security_kubernetes_io_audit": "restricted", "maistra_io_member-of": "istio-system", "pod-security_kubernetes_io_warn-version": "v1.24", "pod-security_kubernetes_io_warn": "restricted", "kubernetes_io_metadata_name": "istio-system", "kiali_io_member-of": "istio-system", "pod-security_kubernetes_io_audit-version": "v1.24" }, "container_image": "registry.redhat.io/openshift-service-mesh/pilot-rhel8@sha256:f12207461e6613649d6c62eea7bdf00829715249068796d59ba2fe99c19948ae", "namespace_name": "istio-system", "pod_name": "istiod-basic-c754cb9cf-xbwqr" }, "viaq_msg_id": "NTk2NWI1ODktZDk1My00MjcyLWE4ZmQtNThjYWRiZDVlYTAw", "level": "info", "openshift": { "sequence": 585, "cluster_id": "938c0ea1-838f-420c-a1ff-2352058b517b" }, "message": "2024-05-17T08:09:03.573347Z\tinfo\tads\tLDS: PUSH for node:istio-ingressgateway-69f8699479-bhshr.istio-system resources:1 size:2.5kB", "docker": { "container_id": "bbb58ae23d1226db7f13d5193a0c5ec4ac6f10d67fee321a8b41361f55498c45" }, "hostname": "hollytest-wbxlb-worker-eastus1-j8r6x", "log_type": "application", "@timestamp": "2024-05-17T08:09:03.573383221+00:00", "pipeline_metadata": { "collector": { "received_at": "2024-05-17T08:09:03.579057+00:00", "name": "fluentd", "inputname": "fluent-plugin-systemd", "version": "1.16.2 1.6.0", "ipaddr4": "10.0.4.136" } } }, "fields": { "@timestamp": [ "2024-05-17T08:09:03.573Z" ], "pipeline_metadata.collector.received_at": [ "2024-05-17T08:09:03.579Z" ] }, "highlight": { "message": [ "2024-05-17T08:09:03.573347Z\tinfo\tads\tLDS: @kibana-highlighted-field@PUSH@/kibana-highlighted-field@ for node:istio-ingressgateway-69f8699479-bhshr.istio-system resources:1 size:2.5kB" ], "kubernetes.namespace_name": [ "@kibana-highlighted-field@istio-system@/kibana-highlighted-field@" ] }, "sort": [ 1715933343573 ] } ------------------------------------------ Vector JSON: { "_index": "app-000001", "_type": "_doc", "_id": "YzRkZWI0N2UtYmUxMS00YWI4LTg2MzAtYmY3MWVkZTc3ZmI1", "_version": 1, "_score": null, "_source": { "kubernetes": { "pod_ip": "10.131.1.143", "annotations": { "seccomp.security.alpha.kubernetes.io/pod": "runtime/default", "prometheus.io/port": "15014", "sidecar.istio.io/inject": "false", "ambient.istio.io/redirection": "disabled", "prometheus.io/scrape": "true", "openshift.io/scc": "restricted-v2", "k8s.ovn.org/pod-networks": "{\"default\":{\"ip_addresses\":[\"10.131.1.143/23\"],\"mac_address\":\"0a:58:0a:83:01:8f\",\"gateway_ips\":[\"10.131.0.1\"],\"ip_address\":\"10.131.1.143/23\",\"gateway_ip\":\"10.131.0.1\"}}", "k8s.v1.cni.cncf.io/network-status": "[{\n \"name\": \"ovn-kubernetes\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.131.1.143\"\n ],\n \"mac\": \"0a:58:0a:83:01:8f\",\n \"default\": true,\n \"dns\": {}\n}]" }, "pod_owner": "ReplicaSet/istiod-basic-5794cf67bb", "namespace_labels": { "pod-security_kubernetes_io_audit": "restricted", "maistra_io_member-of": "istio-system", "pod-security_kubernetes_io_warn-version": "v1.24", "pod-security_kubernetes_io_warn": "restricted", "kubernetes_io_metadata_name": "istio-system", "kiali_io_member-of": "istio-system", "pod-security_kubernetes_io_audit-version": "v1.24" }, "container_image": "registry.redhat.io/openshift-service-mesh/pilot-rhel8@sha256:f12207461e6613649d6c62eea7bdf00829715249068796d59ba2fe99c19948ae", "labels": {}, "namespace_name": "istio-system", "pod_name": "istiod-basic-5794cf67bb-7kt4g", "container_name": "discovery", "flat_labels": [ "istio=istiod", "pod-template-hash=5794cf67bb", "app=istiod", "istio_io_rev=basic", "maistra-control-plane=istio-system", "sidecar_istio_io_inject=false" ], "namespace_id": "1dcdcbf8-9f58-456a-9a6f-21e00fdbd669", "pod_id": "d2f7d700-cd5a-4ed5-848e-14d9825fe264", "container_id": "cri-o://6e4df9af8a5eaba793eee288a867abb15fcacc98dee530422f2bf63c5d615ef0" }, "hostname": "hollytest-wbxlb-worker-eastus3-nh85b", "log_type": "application", "@timestamp": "2024-05-17T07:45:43.319246645Z", "level": "info", "openshift": { "sequence": 139860, "cluster_id": "938c0ea1-838f-420c-a1ff-2352058b517b" }, "message": "2024-05-17T07:45:43.319208Z\tinfo\tads\tSDS: PUSH request for node:istio-ingressgateway-5bdb858658-dnnqh.istio-system resources:1 size:2.8kB cached:1/1" }, "fields": { "@timestamp": [ "2024-05-17T07:45:43.319Z" ] }, "highlight": { "kubernetes.namespace_name": [ "@kibana-highlighted-field@istio-system@/kibana-highlighted-field@" ] }, "sort": [ 1715931943319 ] }
Why does the customer need this? (List the business requirements)
cx would like to make less change to their rsyslog conf as much as possible(keep their old setting with fluentd) and asked whether change from vector JSON can be made to let them use old setting with fluentd to work well with vector as well.
List any affected packages or components.
like for instance host:
Fluentd: kubernetes:
Vector: kubernetes:{} hostname: