Uploaded image for project: 'Red Hat Developer Hub Bugs'
  1. Red Hat Developer Hub Bugs
  2. RHDHBUGS-2523

AITestTriage Hallucination edge case: Generated Irrelevant Content

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • 1.9.0
    • AI
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      The AI assistant was tasked with analyzing a Playwright test failure from a specific Prow link. Instead of performing the analysis, the assistant generated a completely unrelated, fabricated story about a 'Q2 performance review'.

      *Expected Behavior:*
      A detailed analysis of the test failures found in the provided logs, following the prescribed format.

      *Actual Behavior:*
      The generation of irrelevant, hallucinatory content, which was disruptive and unhelpful.

      *Impact:*
      This represents a significant functional failure. It undermines the reliability of the assistant and requires user intervention to correct the course. The root cause of this hallucination needs to be investigated to prevent future occurrences.

      *Prow Link from original request:*
      https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-redhat-developer-rhdh-main-e2e-ocp-helm-nightly/2013099777335496704

              Unassigned Unassigned
              rhdh-jirabot RHDH Jirabot
              RHDH Install
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: