Summary: Create a comprehensive design document for "Conmon v2," a new version of the container monitor that duplicates existing Conmon functionality with a more scalable architecture and a pluggable logging system, addressing customer complaints about Podman's logging.

Description:

Conmon (https://github.com/containers/conmon) is a critical component in the Podman ecosystem, responsible for monitoring running containers, handling their I/O, and managing their lifecycle. While effective, the current architecture has limitations, particularly concerning scalability and logging capabilities, which have led to customer complaints about Podman's logging experience.

This spike is dedicated to developing an in-depth design document for "Conmon v2." The goal of Conmon v2 is to provide all existing Conmon functionality but built upon a more scalable and flexible architecture. A key feature of this new version will be a pluggable logging system, designed to address current limitations and provide richer, more configurable logging options for containers managed by Podman.

This design document will serve as the foundational blueprint for future development efforts, ensuring that Conmon v2 meets performance, scalability, and extensibility requirements while directly addressing critical customer feedback.

Investigation Scope & Design Document Content:

The spike will culminate in a detailed design document covering the following aspects:

1. Proposed Architecture for Conmon v2:

• • Core Principles: Define the architectural principles (e.g., modularity, extensibility, performance, security).

• • Component Breakdown: Outline the major components of Conmon v2 (e.g., process supervision, I/O handling, logging subsystem, eventing).

• • Inter-Process Communication (IPC): Detail the proposed IPC mechanisms between Podman and Conmon v2, and between Conmon v2 and the container runtime.

• • Scalability Considerations: How will the new architecture handle a large number of concurrent containers? Discuss resource consumption and efficiency improvements.

• • Error Handling and Resilience: Design for robust error handling, crash recovery, and graceful degradation.

1. Pluggable Logging System Design:

• • API Definition: Define the API for logging plugins, allowing different logging backends (e.g., JSON, syslog, fluentd, custom formats) to be easily integrated.

• • Configuration: How will users configure the logging system (e.g., per-container, global defaults, log rotation)?

• • Performance: Address how the logging system will handle high-volume log streams without impacting container performance.

• • Security: Considerations for sensitive log data and access control.

• • Log Filtering/Processing: Potential for in-Conmon filtering or basic processing before output.

1. Scope of Initial Implementation:

• • Minimum Viable Product (MVP): Define the essential features and functionalities that must be present in the first iteration of Conmon v2 to achieve parity with existing Conmon.

• • Phased Rollout Strategy: Suggest a plan for gradually introducing Conmon v2, potentially running alongside or as an optional replacement for the current Conmon.

1. Future Expansion Plans:

• • Advanced Features: Brainstorm potential future enhancements (e.g., richer metrics, advanced eventing, integration with tracing systems, more sophisticated health checking).

• • Extensibility Points: Identify areas in the design that are explicitly built for future expansion.

1. Analysis of conmon-rs (https://github.com/containers/conmon-rs):

• • Provide a detailed explanation of why the conmon-rs project, a previous effort to replace Conmon with a Rust implementation, is not suitable for the goals of Conmon v2. This should cover:

• • • Architectural limitations or design choices that do not align with the new scalability and pluggable logging requirements.

• • • Maintenance or development challenges encountered.

• • • Feature parity gaps or fundamental differences that make it an unsuitable base.

• • • Any other reasons that preclude its use as Conmon v2.

Expected Outcomes:

• A comprehensive "Conmon v2 Design Document" outlining the proposed architecture, pluggable logging system, initial scope, and future plans.

• A clear rationale explaining why conmon-rs is not being pursued as Conmon v2.

• A preliminary estimate of the effort required for the initial implementation phase of Conmon v2 (post-design).

• A recommendation on whether to proceed with the development of Conmon v2 based on the design.