Elasticsearch output index string should be prefixed [Done]

Need to clarify some points in the proposal (will update proposal) There is no change of intent, but the intent needs clarification. my not be completely clear.

Index naming - requires code fix

The original intent has not changed, but it got lost in translation:

elasticsearch.indexName or elasticsearch.indexKey provides a sting that is included in the index name. We prefix it with "app-" or "infra-" to follow our index model. We have to do that to distinguish JSON app vs. infra logs.

We need to implement and document that clearly.

We are not introducing new features for external elasticsearch. We are solving the JSON-ES problem by extending our index model. We have always required external ES to follow our index model, they will have to follow this extension as well.

The feature as-is will easy for users of the default ES. We have never provided any assistance to users setting up an external ES other than documenting requirements, that does not change in this release.

Schema vs. Index

The original proposal had a JSON "schema" the final proposal has elasticsearch.index.

There is no requirement to segregate different types of JSON for any other store or forwarding target. This is a unique bug/misfeature of elasticsearch. The forwarder should not even be responsible for this, we hacked it because we couldn't figure out an ES solution.

Therefore we must contain the hack to the elasticsearch output so it does not infect other output types.

defaultOutput settings.

defaultOutput is a way to over-ride type specific settings for the default output. For this release it works only for elasticsearch but in future we will finish the feature to cover any output type that we may use as the default output types.

It does not allow the user to override generic settings (URL, Secret...) because then it would not be the default output. Make a regular output for that.

defaultOutput is not an ES or JSON-specific feature. It is introduced to solve a problem caused by ES, but other output types will have related requirements, e.g. to modify labels on the default loki output.