Klusterlet & Add-on Agent Lifecycle

Problem Statement

Today, whenever ACM (MultiClusterHub operator) is upgraded, not only is the hub operator upgraded; but a fleet-wide upgrade of all Klusterlet and add-ons are initiated and performed simultaneously. Customers, especially in services and edge scenarios, need finer grained control over the upgrade process to minimize disruption and prevent problems getting introduced into their production environments that would impact their end-users.

Guiding Use Case Exploration Document

Roadmap

Phase 0 (RHACM 2.7-2.8 / MCE 2.2-2.3)

• ACM-1493 Use Case exploration and prioritization (Use Case Exploration)
• ACM-1493 Explore detaching the RHACM hub control plane versions from the managed cluster agent versions.
• ACM-1961 Investigate enhancements to ManagedClusterAddOn API to allow for independent add-on lifecycle and versioning.  Submit proposal to Open Cluster Management community.
• ACM-2476 Add-on owners investigate scope of API implications for potential differing API versions on hub and managed cluster.
• Investigation of upgrade experience (stability, time, etc) at scale for Klusterlet and add-ons.
• ACM-4478 Investigation of Klusterlet lifecycle requirements.

Phase 1 (RHACM 2.9 / MCE 2.4)

• ACM-2079 Implement Klusterlet lifecycle enhancements
• ACM-2078 Add-on installation strategy supported
• ACM-3437 Implement ManagedClusterAddOn API lifecycle enhancements
• ACM-2475 MCH/MCE Installer allows for the hub control plane and agent lifecycles to be separated.
• ACM-2476 Add-on owners tolerate the potential differing API versions on hub and managed cluster.
• ACM-5648 Deprecation of KlusterletAddOnConfig API
• ACM-5651 Placement - support progressive rollout strategies
• ACM-5650 Determine proper API entry point for ACM rolling upgrade
• ACM-2478 Initial API for rolling upgrade is established (architecture, design, prototyping)
  • Group clusters for upgrade
  • Allow desired version to be configured
  • Status and feedback on upgrade progression
  • Date/time information on upgrade
  • Code instrumentation and metrics
• Socialize future direction with QE and CEE

Phase 2 (RHACM 2.10 / MCE 2.5 - Tech Preview) (Proposing...)

• Add-on owners implement ManagedClusterAddOn lifecycle enhancements
  • Template issue:  https://issues.redhat.com/browse/ACM-2477 
    • Each component/add-on owner needs to clone this template for each add-on they own.
• ACM-2479 Rolling upgrade API will support:
  • Time window
  • Concurrency parameter
  • Fine grained control - success thresholds
  • Out of the box alerting rules
• Future support agreed upon with QE and CEE
  • Test automation strategy initiated

Phase 3 (RHACM 2.11 / MCE 2.6 - GA) (Proposing...)

• ACM-5649 Removal of KlusterletAddOnConfig API
• Support for agent version rollback
• Full E2E test automation coverage

Further consideration needed for:

• User Experience / User Interface support
• Pre-flight checks