-
Story
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
None
[Testing] Integration Testing with oc-mirror
Summary
Validate the end-to-end workflow with oc-mirror v2 performing multi-architecture filtered mirroring. Ensure digest preservation and sparse manifest handling works correctly in real-world disconnected OpenShift deployment scenarios.
Acceptance Criteria
- [ ] oc-mirror can detect Quay sparse manifest support via capability endpoint
- [ ] oc-mirror can perform architecture-filtered mirroring to Quay
- [ ] Original manifest list digest is preserved after filtered mirroring
- [ ] Filtered architectures are correctly stored in Quay
- [ ] Non-filtered architectures are not stored (storage savings verified)
- [ ] OpenShift can pull images using original digest references
- [ ] Error handling works correctly for missing architectures
- [ ] Performance metrics (storage savings) are measurable
- [ ] Backwards compatibility with non-sparse registries verified
Technical Requirements
Test Environment Setup
Requirements:
- Quay instance with FEATURE_SPARSE_INDEX=True configured
- oc-mirror v2 with sparse manifest support
- Source registry with multi-architecture images
- Test namespace/repository for mirroring
- Metrics collection for storage comparison
Test Scenarios
Scenario 1: Basic Architecture-Filtered Mirror
# Configure oc-mirror to use sparse manifest support # Mirror only amd64 from a multi-arch image # 1. Create imageset config for filtered mirror cat > imageset-config.yaml <<EOF kind: ImageSetConfiguration apiVersion: mirror.openshift.io/v1alpha2 mirror: platform: architectures: - amd64 operators: - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14 packages: - name: advanced-cluster-management EOF # 2. Run oc-mirror with sparse mode oc-mirror --config imageset-config.yaml \ docker://quay.example.com/test-mirror \ --sparse-index # 3. Verify results # - Original digest preserved # - Only amd64 manifests present # - Manifest list is marked as sparse
Scenario 2: Multiple Architecture Filter
# Mirror amd64 and arm64 (common cloud/edge combo)
# 1. Configure for multiple architectures
cat > imageset-config.yaml <<EOF
kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
platform:
architectures:
- amd64
- arm64
EOF
# 2. Run mirror and verify both architectures present
# 3. Verify ppc64le and s390x are NOT present
# 4. Verify manifest list digest matches source
Scenario 3: Digest Preservation Verification
# Test script to verify digest preservation import requests import hashlib def verify_digest_preservation(source_registry, dest_registry, image, tag): """Verify manifest list digest is preserved after sparse mirror.""" # Get source manifest list src_response = requests.get( f"https://{source_registry}/v2/{image}/manifests/{tag}", headers={"Accept": "application/vnd.oci.image.index.v1+json"} ) src_digest = hashlib.sha256(src_response.content).hexdigest() # Get destination manifest list dest_response = requests.get( f"https://{dest_registry}/v2/{image}/manifests/{tag}", headers={"Accept": "application/vnd.oci.image.index.v1+json"} ) dest_digest = hashlib.sha256(dest_response.content).hexdigest() assert src_digest == dest_digest, \ f"Digest mismatch: source={src_digest}, dest={dest_digest}" print(f"✓ Digest preserved: sha256:{src_digest}") # Run verification verify_digest_preservation( "registry.redhat.io", "quay.example.com", "test-mirror/advanced-cluster-management", "v2.9.0" )
Scenario 4: OpenShift Pull Test
# Verify OpenShift can pull using original digest
# 1. Deploy pod using digest reference
cat > test-pod.yaml <<EOF
apiVersion: v1
kind: Pod
metadata:
name: test-sparse-pull
spec:
containers:
- name: test
image: quay.example.com/test-mirror/image@sha256:abc123...
EOF
kubectl apply -f test-pod.yaml
# 2. Verify pod starts successfully
kubectl wait --for=condition=Ready pod/test-sparse-pull --timeout=60s
# 3. Verify correct architecture was pulled
kubectl exec test-sparse-pull -- uname -m
Scenario 5: Storage Savings Measurement
# Compare storage usage between full and sparse mirrors
def measure_storage_savings(full_mirror_repo, sparse_mirror_repo):
"""Compare storage between full and sparse mirrors."""
# Get repository storage from Quay API
full_size = get_repo_storage(full_mirror_repo)
sparse_size = get_repo_storage(sparse_mirror_repo)
savings = (full_size - sparse_size) / full_size * 100
print(f"Full mirror storage: {full_size / (1024**3):.2f} GB")
print(f"Sparse mirror storage: {sparse_size / (1024**3):.2f} GB")
print(f"Storage savings: {savings:.1f}%")
# Epic requires 50%+ savings for single architecture
assert savings >= 50, f"Expected 50%+ savings, got {savings:.1f}%"
def get_repo_storage(repo):
"""Get total storage for a repository."""
response = requests.get(
f"https://quay.example.com/api/v1/repository/{repo}",
headers={"Authorization": f"Bearer {token}"}
)
return response.json().get("quota_report", {}).get("configured_quota", 0)
Scenario 6: Error Handling for Missing Architecture
# Verify appropriate error when pulling missing architecture # 1. Mirror only amd64 oc-mirror --config amd64-only.yaml docker://quay.example.com/test-mirror # 2. Attempt to pull arm64 (should fail gracefully) podman pull --arch arm64 quay.example.com/test-mirror/image:tag # Expected error: # Error: manifest unknown: Could not find manifest for architecture arm64
Scenario 7: Backwards Compatibility
# Verify non-sparse registries still work
# 1. Disable sparse support on Quay
# FEATURE_SPARSE_INDEX = False
# 2. Attempt filtered mirror
oc-mirror --config filtered.yaml docker://quay.example.com/test-mirror
# 3. Verify oc-mirror falls back to full mirror
# OR
# 3. Verify oc-mirror detects lack of support and fails gracefully
Test Data Requirements
Use well-known multi-architecture images for testing:
- registry.access.redhat.com/ubi8/ubi:latest (amd64, arm64, ppc64le, s390x)
- OpenShift operator images with multiple architectures
- Custom test images with known architecture content
Automated Test Suite
File: test/integration/test_sparse_manifest_oc_mirror.py (new file)
import pytest import subprocess import requests class TestOcMirrorIntegration: """Integration tests for oc-mirror with sparse manifest support.""" @pytest.fixture def quay_with_sparse(self): """Configure Quay with sparse manifest support enabled.""" # Setup Quay with FEATURE_SPARSE_INDEX=True yield # Cleanup def test_capability_detection(self, quay_with_sparse): """Test oc-mirror can detect sparse support.""" response = requests.get( "https://quay.example.com/api/v1/registry/capabilities" ) assert response.json()["sparse_manifests"]["supported"] == True def test_filtered_mirror_single_arch(self, quay_with_sparse): """Test mirroring single architecture.""" result = subprocess.run([ "oc-mirror", "--config", "test-amd64-only.yaml", "--sparse-index", "docker://quay.example.com/test" ], capture_output=True) assert result.returncode == 0 def test_digest_preserved(self, quay_with_sparse): """Test original digest is preserved.""" # Implementation as shown above def test_missing_arch_error(self, quay_with_sparse): """Test pulling missing architecture returns error.""" result = subprocess.run([ "podman", "pull", "--arch", "ppc64le", "quay.example.com/test/image:tag" ], capture_output=True) assert result.returncode != 0 assert b"manifest unknown" in result.stderr def test_storage_savings(self, quay_with_sparse): """Test 50%+ storage savings for single arch.""" # Implementation as shown above
Implementation Notes
Test Environment Options
- Local development: Use Podman/Docker with local Quay instance
- CI/CD integration: Use ephemeral Quay instances in OpenShift
- Manual testing: Use dedicated QE environment
Coordination with oc-mirror Team
- Align on capability detection endpoint format
- Confirm oc-mirror v2 sparse manifest support is available
- Share test results and edge cases
Metrics to Collect
- Storage usage comparison (full vs sparse)
- Mirror operation time (should be faster with fewer manifests)
- Pull success/failure rates by architecture
- Digest verification results
Known Limitations
- oc-mirror v2 required (v1 may not support sparse)
- Skopeo version must support individual manifest copying
- Source registry must be accessible during test
Dependencies
- Story 03: Mirror worker architecture filtering must be complete
- Story 06: UI architecture filter for configuration
- Story 07: UI sparse visualization for verification
- Story 04: API capability detection endpoint
Testing Requirements
Pre-requisites
- [ ] oc-mirror v2 with sparse support available
- [ ] Test Quay instance configured
- [ ] Test image data available in source registry
- [ ] Network connectivity between components
Test Execution Plan
- Unit Tests: Run automated test suite
- Manual Verification: Execute scenarios manually
- Performance Testing: Measure storage savings
- Edge Cases: Test error handling and fallbacks
Documentation Updates
After testing, update:
- User documentation with tested workflows
- Troubleshooting guide with common issues found
- Release notes with verified capabilities
Definition of Done
- [ ] All test scenarios pass
- [ ] Storage savings meet 50%+ target
- [ ] Digest preservation verified
- [ ] OpenShift pull workflow validated
- [ ] Error handling verified
- [ ] Backwards compatibility confirmed
- [ ] Test results documented
- [ ] oc-mirror team sign-off
- [ ] QE test plan reviewed and approved