Proposed title of this feature request

Enable option for debugging Vector for troubleshooting

What is the nature and description of the request?

In fluentd, it was very easy to debug it. Some examples:

• Enable debugging option (this required to move to Unmanaged)
• For checking the content of the data log forwarded, it was easy to review the content of the buffer files and if the content of it was the expected: metadata
• Number of buffer files on the queue indicating delays or problems to log forwarding to an specific output (probably, this can be observed in the metrics now)
• Verify when done an specific configuration, for example, enabling json, really the content is parsed as expected and sent to the expected indice defined

As Vector is now implemented to work in memory, we are blind from the point of view of troubleshooting and helping to make easy to understand and explain why the things are happening.

Then, it should be desired to have an option to enable debug in an easy way for doing troubleshooting and identify possible problems and even better if this option can be in a "Managed" status.

Also, some instructions to dump the data that Vector is holding in the memory for its revision.

Why does the customer need this? (List the business requirements)

Any kind of troubleshooting related to Vector and having more details about what's happening, content log forwarded, indices where the content is sent, etc.

List any affected packages or components.

• ClusterLogging - ClusterLogForwarder API

• Vector

NOTES:

From grooming Dec19.  Three parts to this request:  
1. Add the ability to set vector's log level (debug) in a Managed status
2. Add the ability to do a "core dump" of vector data/state
3. Add feature similar to vector tap - (delivered)

The exact needed is what vector tap provides: https://vector.dev/guides/level-up/vector-tap-guide/ and also, https://vector.dev/guides/level-up/troubleshooting/