Story (Required)

As a platform administrator trying to control LLM API usage and costs I want team-based access control for LLM analysis using PAC's existing policy system so that only authorized users can trigger AI analysis and send data to external APIs

This feature integrates LLM analysis with Pipelines-as-Code's existing policy system (similar to pull_request and ok_to_test actions). Administrators can define which GitHub/Gitea teams are allowed to trigger LLM analysis, preventing unauthorized API usage, controlling costs, and ensuring compliance with data sharing policies.

Background (Required)

Currently, LLM analysis has no access control. This creates risks:

• Anyone with PR access can trigger expensive LLM API calls
• External contributors in public repos can consume LLM budgets
• No way to restrict who can send logs/code to external LLM providers
• Compliance concerns about data leaving organization
• No audit trail of who triggered which analyses

PAC already has a team-based policy system for controlling pull_request and ok_to_test actions. Extending this to LLM analysis provides consistent access control.

Related documentation:

• Current policy system: docs/content/docs/guide/policy.md
• LLM analysis: docs/content/docs/guide/llm-analysis.md

Out of scope

• Role-Based Access Control (RBAC) beyond team membership
• Fine-grained permissions (e.g., per-model or per-provider access)
• User quotas or individual budget limits
• Integration with external authorization systems (OAuth, LDAP)
• Approval workflows for analysis requests

Approach (Required)

High-level technical approach:

Add new llm_analysis action to the policy system (similar to existing pull_request and ok_to_test)

Extend Repository.spec.settings.policy to include llm_analysis field with list of allowed teams

Add optional per-role policy.required_teams override for granular control

When analysis is triggered (automatically or manually)

*Check if triggering user belongs to teams in settings.policy.llm_analysis

* Check if user is in OWNERS file (bypass policy, consistent with PAC behavior)

*Check role-specific required_teams if configured

* Log authorization decision for audit

Support manual triggering via comment commands (e.g., /pac analyze)

Respect policy for both automatic (on-failure) and manual (on-demand) triggers

Provide clear error messages when authorization fails

The feature should integrate seamlessly with existing PAC policy infrastructure.

Dependencies

• Existing PAC policy system and team membership resolution
• LLM analysis infrastructure
• Repository CRD must support llm_analysis policy field
• Comment command parsing infrastructure (for manual triggers)
• Provider APIs for fetching team membership (GitHub, Gitea)

Acceptance Criteria (Mandatory)

Given a Repository with policy.llm_analysis: ["ci-admins"], When a user from ci-admins team triggers analysis, Then the analysis runs successfully

Given the same policy, When a user NOT in ci-admins team triggers analysis, Then the request is denied with a clear error message

Given a user listed in the OWNERS file, When that user triggers analysis, Then the analysis runs regardless of team membership (OWNERS bypass)

Given a role with policy.required_teams: ["senior-engineers"], When a user from ci-admins (global policy) but NOT senior-engineers triggers that role, Then the request is denied

Given automatic analysis on failure, When the PR author is not authorized, Then the analysis does not run and logs the authorization failure

Given a user comments /pac analyze on a PR, When authorization succeeds, Then LLM analysis is triggered manually

Given authorization decisions are made, When viewing audit logs, Then each decision includes user, team membership, policy used, and result

Edge cases to consider:

• Users belonging to multiple teams (any match grants access)
• Team membership caching and refresh timing
• Private repositories vs public repositories
• Users with no team affiliation
• OWNERS file format variations
• Policy not configured (default behavior: allow all? deny all?)
• Combining global and per-role policies