-
Epic
-
Resolution: Unresolved
-
Major
-
None
-
deletionOrder fix
-
XL
-
False
-
-
False
-
-
In Progress
-
GITOPS-8789 - AppSet Progressive Sync GA (in upstream)
-
0% To Do, 50% In Progress, 50% Done
-
-
Story (Required)
- When an ApplicationSet with deletionOrder: Reverse is deleted, Kubernetes garbage collection (due to blockOwnerDeletion: true on owner references) can set DeletionTimestamp on all dependent Applications synchronously during the delete API call, before the ApplicationSet controller reconciles. This can cause all Applications to appear as "deleting" simultaneously in the UI, bypassing the intended reverse deletion order.
- Upstream issue created https://github.com/argoproj/argo-cd/issues/26612
Background and Approach (Required)
- During e2e testing with aggressive event processing intervals (ARGOCD_CLUSTER_CACHE_EVENTS_PROCESSING_INTERVAL=1ms), a race condition was observed.
- When an ApplicationSet with deletionOrder: Reverse is deleted, Kubernetes garbage collection (due to blockOwnerDeletion: true on owner references) can set DeletionTimestamp on all dependent Applications synchronously during the delete API call, before the ApplicationSet controller reconciles. This can cause all Applications to appear as "deleting" simultaneously in the UI, bypassing the intended reverse deletion order.
Why It's Hard to Observe:
- Kubernetes GC cascades deletion synchronously in the API server, making it hard to intercept
- The behavior is more visible in e2e tests with 1ms event processing intervals
- In manual testing, the UI refresh timing or observation window may mask the intermediate state
- The controller's reverse deletion logic still runs, but all Applications may already have DeletionTimestamp set
Out of Scope
- Downstream e2e tests for progressive sync
- Performance, load, or stress testing of progressive sync.
- Refactoring or redesign of the progressive sync feature's core logic.
- Adding unit or integration tests for specific components of progressive sync (this story focuses on the end-to-end flow).
Dependencies
- No dependecies
Acceptance Criteria (Mandatory)
- issue is fixed
- e2e test for deletionOrder pass
Definition of Done
- Code Complete:
- All code has been written, reviewed, and approved.
- Tested:
- Unit tests have been written and passed.
- Ensure code coverage is not reduced with the changes.
- Integration tests have been automated.
- System tests have been conducted, and all critical bugs have been fixed.
- Tested and merged on OpenShift either upstream or downstream on a local build.
- Documentation:
- User documentation or release notes have been written (if applicable).
- Build:
- Code has been successfully built and integrated into the main repository / project.
- Midstream changes (if applicable) are done, reviewed, approved and merged.
- Review:
- Code has been peer-reviewed and meets coding standards.
- All acceptance criteria defined in the user story have been met.
- Tested by reviewer on OpenShift.
- Deployment:
- The feature has been deployed on OpenShift cluster for testing.
- clones
-
GITOPS-7210 Add e2e tests for progressive sync upstream
-
- Release Pending
-