-
Story
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
False
-
-
False
-
-
Problem
Currently, the chart and documentation may become out of sync, and there's no automated validation to ensure:
- Documentation accurately reflects chart configuration
- No 'latest' tags are introduced in future changes
- Custom image workflows function correctly
- Chart automation works as expected
Solution
Enhance the existing pytest testing framework to include comprehensive validation of chart consistency, documentation accuracy, and automation workflows.
Acceptance Criteria
- [ ] Create tests to validate documentation against chart configuration
- [ ] Add tests to ensure no 'latest' tags are present in templates/values
- [ ] Implement integration tests for custom image workflow validation
- [ ] Add tests for chart update automation workflow
- [ ] Create smoke tests for deployed applications with new images
- [ ] Implement tests for GitHub Pages helm repository functionality
- [ ] Add validation tests for oc-mirror compatibility
- [ ] Document testing procedures and coverage
- [ ] Integrate new tests into CI/CD pipeline
- [ ] Create test reports and coverage metrics
Test Categories
- Documentation Sync Tests
- Validate values.yaml documentation matches actual values
- Check README instructions against chart capabilities
- Verify example configurations work as documented
- Image Versioning Tests
- Scan for any 'latest' tags in chart files
- Validate all images use semantic versioning
- Check image availability in configured registries
- Automation Integration Tests
- Test GitHub workflow triggers and outputs
- Validate chart update automation
- Verify helm repository functionality
- Disconnected Environment Tests
- Test oc-mirror compatibility
- Validate chart templates without cluster connectivity
- Verify no 'lookup' commands remain
Definition of Done
- Comprehensive test coverage for all epic objectives
- Tests integrated into CI/CD pipeline
- Documentation and chart remain synchronized
- Regression prevention for lookup commands and latest tags
- Automation workflows are validated and reliable