Concurrent Backup Design Implementation

Based on Velero PR: #8991

Original Issue: #8899

Overview

This epic implements concurrent backup processing capability in OADP/Velero, allowing multiple backups to run simultaneously while preventing conflicts and ensuring data integrity.

Background

Currently, Velero processes backups sequentially, which can lead to performance bottlenecks and delayed backup completion times. This design introduces concurrent backup processing with intelligent conflict detection and resolution.

Key Features

Conflict Detection Levels

_ _Namespace-level conflict detection:* Prevents backups of the same workloads from running concurrently
_ _ItemBlock-level conflict detection:* Allows fine-grained resource-level conflict prevention
_ _Configuration flexibility:* Configurable flags to choose between namespace-level or ItemBlock-level detection

Safety Mechanisms

• Prevents concurrent backups for serial workloads (e.g., hourly backups for same workload)
• Blocks concurrent operations on resources requiring snapshot/flush/freeze operations
• Maintains data integrity during concurrent operations

Implementation Considerations

Namespace-Level Detection

• Immediately blocks backups targeting the same namespaces
• Prevents logical conflicts at workload level
• Safer approach for critical workloads

ItemBlock-Level Detection

• Allows backups with overlapping namespaces to run concurrently
• Only prevents conflicts at individual resource level
• Enables higher concurrency for cluster-scoped resources
• Handles resources selected by selectors

Configuration Options

The implementation should provide configuration flags to allow users to choose between:

• Strict namespace-level conflict prevention (safer, lower concurrency)
• Granular ItemBlock-level conflict prevention (higher concurrency, more complex)

Target Release

• Design completion: Before Velero 1.17 release
• Implementation target: Velero 1.18 / OADP 1.6.0

Risk Mitigation

• Backup engine isolation from upper-level modules
• Comprehensive conflict detection to prevent data corruption
• Fallback mechanisms for overlapping backup scenarios