Story (Required)

As a DevOps engineer trying to *optimize CI/CD resource consumption and
provide early feedback_ I want _LLM analysis to run before PipelineRun
execution with optional confidence-based gating*

*This feature enables AI-powered analysis of pull requests at creation time,
before any pipeline resources are consumed. The LLM analyzes the PR content,
commit changes, and code diff to provide immediate feedback and optionally gate
pipeline execution based on a confidence score. This improves developer
experience through faster feedback and reduces infrastructure costs by skipping
unnecessary pipeline runs for trivial changes.*

Background (Required)

Currently, the LLM analysis feature runs after PipelineRun completion
(typically on failures). This means:

• Developers wait for the entire pipeline to complete before receiving AI
  insights
• Cluster resources are consumed even for changes that could be identified as
  low-risk or trivial
• No opportunity to skip expensive CI runs based on change analysis

This story introduces a pre-execution analysis phase where the LLM can
analyze changes before the pipeline starts, provide early feedback, and
optionally gate execution based on confidence scoring.

Out of scope

• Implementation of specific confidence scoring algorithms (left to LLM
  provider responses)
• Integration with external approval systems or manual review workflows
• Historical analysis or machine learning to improve confidence scoring over
  time (covered in separate story for "Analysis History and Learning")
• Cost tracking and budget controls (covered in separate story)

Approach (Required)

High-level technical approach:

Add a new trigger phase concept to the AnalysisRole configuration: trigger_phase with values pre-execution or post-execution (default)

Introduce a confidence_gate configuration block that includes

*Enable/disable flag

* Minimum confidence score threshold

* Action to take when threshold not met (skip, require-approval, etc.)

Modify the analysis workflow to run configured pre-execution roles at PipelineRun creation time

Parse LLM responses to extract confidence scores (either structured or from free-text responses)

Implement gating logic that can prevent/delay PipelineRun execution based on confidence threshold

Ensure results are posted to configured output destinations (PR comments, etc.) regardless of gate decision

The feature should be fully backward compatible - existing configurations continue to work as post-execution analysis.

Dependencies

• Existing LLM analysis infrastructure (OpenAI/Gemini clients, context assembly, CEL evaluation)
• Repository CRD must support new configuration fields for trigger_phase and confidence_gate
• May depend on PipelineRun admission webhook or controller modifications to support pre-execution gating

Acceptance Criteria (Mandatory)

Given a Repository with an LLM role configured with trigger_phase: "pre-execution", When a pull request is opened, Then the LLM analysis executes before the PipelineRun starts

Given a pre-execution role with confidence_gate.enabled: true and minimum_score: 70, When the LLM returns a confidence score >= 70, Then the PipelineRun executes normally

Given a pre-execution role with confidence gating enabled, When the LLM returns a confidence score below the threshold, Then the PipelineRun is skipped/gated according to the configured action

Given a pre-execution analysis completes, When the result is available, Then the analysis is posted to the configured output destination (e.g., PR comment) with the confidence score visible

Given multiple roles with different trigger phases, When an event occurs, Then pre-execution roles run before pipeline start and post-execution roles run after completion

Given a pre-execution analysis fails or times out, When the failure occurs, Then the PipelineRun proceeds normally (fail-safe behavior) and an error is logged

Given a Repository without pre-execution roles configured, When a PR is created, Then existing post-execution behavior continues unchanged (backward compatibility)

Edge cases to consider:

• LLM API timeouts or failures during pre-execution phase
• Parsing confidence scores from both structured JSON and free-text LLM responses
• Handling repositories with both pre and post-execution roles configured
• CEL expression evaluation at pre-execution time (limited context compared to post-execution)
• Concurrent PR updates while pre-execution analysis is running