-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
quay-v3.17.0
-
False
-
-
False
-
-
Summary
When tags become immutable due to immutability policy enforcement (either automatically on push or retroactively when policies are created/updated), *no audit log entries are created* to record these state changes. This creates critical gaps in compliance audit trails and makes it impossible to trace when/why tags became immutable.
Audit Logging Gaps
What IS Logged ✅
Immutability Policy CRUD Operations:
- create_immutability_policy - When policy is created (endpoints/api/immutability_policy.py:103)
- update_immutability_policy - When policy is updated (endpoints/api/immutability_policy.py:149)
- delete_immutability_policy - When policy is deleted (endpoints/api/immutability_policy.py:165, 300)
Manual Tag Immutability Changes:
- change_tag_immutability - When user explicitly toggles tag immutability via REST API (endpoints/api/tag.py:272-278)
What is NOT Logged ❌ (Critical Gaps)
Gap 1: No Audit Log When New Tags Automatically Become Immutable
File: data/model/oci/tag.py:481-498
# Check if tag should be immutable based on policies immutable = False if features.IMMUTABLE_TAGS: from data.model import immutability immutable = immutability.evaluate_immutability_policies( manifest.repository_id, repo.namespace_user_id, tag_name ) created = Tag.create( name=tag_name, repository=manifest.repository_id, manifest=manifest, immutable=immutable, # ← Automatically set, NO LOG! )
Impact:
- When user pushes nginx:latest and it matches policy pattern ^latest$, tag becomes immutable
- *NO audit log entry* showing this happened
- User cannot determine when/why the tag became immutable
- No traceability to which policy caused the immutability
Gap 2: No Audit Log When Existing Tags Become Immutable Due to Policy Creation/Update
File: data/model/immutability.py:161-207, 142-158
def apply_immutability_policy_to_existing_tags(...): """Retroactively apply an immutability policy to existing tags.""" total_marked = 0 while True: tags = _fetch_candidate_tags_batch(namespace_id, repository_id, last_id, batch_size) tag_ids_to_mark = [ tag.id for tag in tags if not tag.immutable and _matches_policy(tag.name, tag_pattern, tag_pattern_matches) ] if tag_ids_to_mark: marked = _mark_tags_immutable_batch(tag_ids_to_mark) # ← NO LOG! total_marked += marked return total_marked def _mark_tags_immutable_batch(tag_ids: list[int]) -> int: """Mark a batch of tags as immutable.""" return ( Tag.update(immutable=True) # ← Direct database UPDATE, NO audit log! .where( Tag.id << tag_ids, Tag.immutable == False, ) .execute() )
When This Happens:
- User creates new immutability policy with pattern ^v.*$
- Function create_namespace_immutability_policy() calls apply_immutability_policy_to_existing_tags() (line 289-294)
- *Could mark hundreds of existing tags as immutable*
- *NO audit log entries* for any of these changes
Impact:
- Administrator creates policy affecting 500 tags
- All 500 tags silently become immutable
- No audit trail showing which tags were affected
- No record of when this happened (beyond the policy creation timestamp)
Affected Code Files
- data/model/oci/tag.py (lines 481-498) - Automatic immutability on tag creation
- data/model/immutability.py (lines 161-207) - Retroactive policy application
- data/model/immutability.py (lines 142-158) - Batch immutability marking
- data/model/immutability.py (lines 269-311) - create_namespace_immutability_policy()
- data/model/immutability.py (lines 314-367) - update_namespace_immutability_policy()
- endpoints/api/tag.py (lines 272-278) - Manual immutability change (already logs correctly)
Audit Trail Gaps Summary
| Scenario | Current Logging | What's Missing |
|---|---|---|
| User creates immutability policy | ✅ create_immutability_policy | ❌ Which existing tags became immutable |
| User updates immutability policy | ✅ update_immutability_policy | ❌ Which tags changed immutability state |
| User pushes new tag matching policy | ❌ None | ❌ Tag created as immutable (no log) |
| User manually toggles tag immutability | ✅ change_tag_immutability | N/A (working correctly) |
Example Audit Gap Scenario
Timeline:
- 2026-03-06 10:00 - Admin creates namespace policy: {"tag_pattern": "^v.*$", "tag_pattern_matches": true}
- Audit Log Entry: create_immutability_policy (namespace: myorg, pattern: ^v.*$)
- Behind the scenes: 847 existing tags (v1.0.0, v2.3.1, etc.) are marked immutable
- Audit Log Entry: ❌ NONE - No record of these 847 tag changes
- 2026-03-06 10:05 - User tries to overwrite v1.0.0 tag
- Error: tag is immutable and cannot be overwritten
- User Investigation: Checks audit logs, sees policy creation but NO record showing this tag became immutable
Compliance & Security Impact
Compliance Issues:
- SOC 2 Audit Trail: Cannot demonstrate when/why tags became immutable
- Change Control: No record of mass immutability changes
- Forensic Investigation: Cannot trace immutability state changes back to policy changes
- ISO 27001 Compliance: Missing audit trail for critical asset state changes
User Experience Issues:
- Users cannot determine why their tag is immutable
- No visibility into which policy caused immutability
- Difficult to troubleshoot unexpected immutable tags
- No way to audit which tags were affected by policy changes
Scale of Impact:
- Single policy creation could affect thousands of tags
- No record of this mass change in audit logs
- Only the policy creation is logged, not the consequences
Recommended Fixes
Fix 1: Log Retroactive Immutability Changes (Batch Summary)
File: data/model/immutability.py:161-207
def apply_immutability_policy_to_existing_tags(...): from data import model as data_model total_marked = 0 affected_tags = [] # Track for audit log while True: tags = _fetch_candidate_tags_batch(...) tag_ids_to_mark = [...] if tag_ids_to_mark: marked = _mark_tags_immutable_batch(tag_ids_to_mark) total_marked += marked # Collect tag names for audit log affected_tags.extend([tag.name for tag in tags if tag.id in tag_ids_to_mark]) last_id = tags[-1].id # ✅ Log batch immutability change if total_marked > 0: from endpoints.api import log_action log_action( "apply_immutability_policy_retroactive", namespace_name, { "namespace": namespace_name, "repo": repository_name if repository_id else None, "tag_pattern": tag_pattern, "tags_affected_count": total_marked, "sample_tags": affected_tags[:10], # First 10 for reference }, repo_name=repository_name if repository_id else None, ) return total_marked
New LogEntryKind Needed:
# In initdb.py or migration LogEntryKind.create(name="apply_immutability_policy_retroactive")
Fix 2: Log Automatic Immutability on Tag Creation
File: data/model/oci/tag.py:481-498
# Check if tag should be immutable based on policies immutable = False matching_policy = None if features.IMMUTABLE_TAGS: from data.model import immutability immutable = immutability.evaluate_immutability_policies( manifest.repository_id, repo.namespace_user_id, tag_name ) # ✅ Get which policy matched (requires enhancing evaluate_immutability_policies) if immutable: matching_policy = immutability.get_matching_policy_info( manifest.repository_id, repo.namespace_user_id, tag_name ) created = Tag.create( name=tag_name, repository=manifest.repository_id, manifest=manifest, immutable=immutable, ) # ✅ Log if tag was created as immutable if immutable and matching_policy: from endpoints.api import log_action log_action( "tag_created_immutable_by_policy", namespace_name, { "namespace": namespace_name, "repo": repository_name, "tag": tag_name, "policy_uuid": matching_policy.get("uuid"), "policy_pattern": matching_policy.get("pattern"), "policy_scope": matching_policy.get("scope"), # "namespace" or "repository" }, repo_name=repository_name, )
New LogEntryKind Needed:
LogEntryKind.create(name="tag_created_immutable_by_policy")
Alternative: Enhanced change_tag_immutability Log
Instead of new log entry kinds, extend existing change_tag_immutability to include policy-driven changes:
log_action(
"change_tag_immutability",
namespace,
{
"username": username or "system", # "system" for policy-driven
"repo": repository,
"tag": tag_name,
"immutable": True,
"previous_immutable": False,
"trigger": "policy", # "manual" or "policy"
"policy_uuid": policy_uuid if policy_driven else None,
"policy_pattern": pattern if policy_driven else None,
},
repo_name=repository,
)
Impact Assessment
Severity: High (Compliance/Audit Issue)
- Compliance Risk: Cannot demonstrate change control for SOC 2, ISO 27001
- User Impact: Confusion about immutability, difficult troubleshooting
- Operational Impact: No visibility into mass immutability changes
- Security Impact: Cannot forensically trace when/why tags became immutable
Affected Users:
- All users with immutability policies (common enterprise feature)
- Compliance-sensitive organizations (finance, healthcare, government)
- Administrators investigating immutability issues
Frequency:
- Occurs every time a tag is pushed that matches a policy
- Occurs every time a policy is created or updated (mass retroactive changes)
- High impact for organizations with active immutability policies
Reproduction Steps
Scenario 1: Tag Creation
- Create namespace immutability policy: pattern ^latest$
- Push new image tag: podman push quay.example.com/org/repo:latest
- Check audit logs for the repository
- Observe: No log entry showing tag became immutable
- Expected: Log entry showing tag created as immutable due to policy
Scenario 2: Retroactive Application
- Create repository with 100 existing tags: v1.0.0, v1.1.0, v2.0.0, etc.
- Create immutability policy: pattern ^v.*$
- Check audit logs
- Observe: Only create_immutability_policy entry, no entries for 100 tags becoming immutable
- Expected: Batch log entry showing 100 tags were marked immutable by policy
Testing Requirements
- Unit test: Verify audit log created when tag matches policy on creation
- Unit test: Verify batch audit log created when retroactively applying policies
- Integration test: Create policy affecting 1000+ tags, verify audit log
- Compliance test: Verify audit trail completeness for SOC 2 requirements
- Performance test: Ensure logging doesn't significantly impact tag push performance
Related Features
- Immutability policies (namespace and repository level)
- Audit logging system
- Tag lifecycle operations
- Compliance and security auditing