Goals

• Specify alternate encodings for log output records such as json, protobuf or raw text.
• Control size of meta-data  in log output records by selecting fields to include or exclude

• • Select pre-defined small, medium or large field sets based on common use patterns.
  • Select specific fields to include or exclude
• Choose the schema (field names an structore) of log records; for exmaple viaq or opentelemetry

Non-Goals

• Not supporting aribtrary re-naming or transformation of meta-data fields, only selection of existing fields
• Not supporting open-ended or user-defined log schema, only pre-defined schema.
• Does not include OTLP protocol outputs (this may implemented separately)
• Does not include new encodings  (e.g. grpc, protobuf) but allows for them to be added later. This epic only covers json and plain text encoding.

Motivation

• Some logs are already structured and carry their own meta-data, users want to consume in the original format.
  • Examples: netflow events, k8s events.
• Users want to reduce the overhead of unused meta-data in log records.
• Users want to forward logs to tools that understand a particular scheme, for example opentelemetry

Alternatives

None.

Acceptance Criteria

Tests to verify:

• Log output includes exactly the fields selected by configuration
• Log output uses the selected schema labels selected by configuration
• Log output uses the selected encoding

Risk and Assumptions

• Custom log records may not be consumable by some targets, for example:
  • openshift log console assumes at least minimal source fields are present.
  • elasticsearch requires special fields for indexing.
  • tools expecting viaq logs cannot consume opentelemetry logs and vice-versa

Documentation Considerations

See LOG-2920.

Open Questions

• Define small/medium/large field sets
• Verify formatting expectations of opentelemetry tools