Description of problem:

OpenShift Container Platform 4 - Cluster Logging is currently configured with fluentd and ClusterLogForwarder to send logs to default elasticsearch and in parallel LokiStack. This all works fine and to enhance LokiStack capabilities, labelKeys were added to the Loki ClusterLogForwarder output specification.

Now, when switching from fluentd to vector as collector, vector immediately enters the CrashLoopBackOff state with the below reported error message.

$ oc logs collector-9q6pn
Defaulted container "collector" out of: collector, logfilesmetricexporter
2023-08-18T14:38:29.580435Z ERROR vector::cli: Configuration error. error=Invalid field path in template "kubernetes.namespace_labels.foo-bar_com_platform-stage" (see https://vector.dev/docs/reference/configuration/template-syntax/)
in `sinks.loki_output_application`

It's though expected that transition between Fluentd and vector can be triggered without such an error being raised and to have vector support and work custom labelKeys

Version-Release number of selected component (if applicable):

OpenShift Container Platform 4 - Cluster Logging 5.7.4

How reproducible:

Always

Steps to Reproduce:

1. Install OpenShift Container Platform 4 - Cluster Logging with default sink configured for elasticsearch and ClusterLogForwarder with Loki and add labelKeys as mentioned above (for example kubernetes.namespace_labels.foo-bar_com_platform-stage).

Actual results:

$ oc logs collector-9q6pn
Defaulted container "collector" out of: collector, logfilesmetricexporter
2023-08-18T14:38:29.580435Z ERROR vector::cli: Configuration error. error=Invalid field path in template "kubernetes.namespace_labels.foo-bar_com_platform-stage" (see https://vector.dev/docs/reference/configuration/template-syntax/)
in `sinks.loki_output_application`

Expected results:

No error being reported and collector to work with vector as it did with fluentd

Additional info: