Uploaded image for project: 'OPCT - OpenShift Provider Compatibility Tool'
  1. OPCT - OpenShift Provider Compatibility Tool
  2. OPCT-236

Conformance suite/review: improve references for test failures

    XMLWordPrintable

Details

    • Epic
    • Resolution: Unresolved
    • Undefined
    • None
    • None
    • Plugins
    • None
    • Conformance suite/review: link references for test failures
    • False
    • Hide

      None

      Show
      None
    • False
    • To Do
    • 0
    • 0% 0%

    Description

      Story:

      • As a user, I would like to be able to identify the failed test, read the logs, and review the test documentation, so I can investigate from my end and eventually fix any issue specific to my deployment/infrastructure
      •  

      Goal:

      • Provide to the user the documentation for failed tests, dynamically based on the failures that happened in the runtime
      • Provide the user a way to quickly read the logs for a failed job

      Description:

      OPCT provides an interface to orchestrate existing conformance suites provided by openshift-tests utility. All the e2e tests are defined by that utility. The failed tests can be dynamically for each environment.

      Currently OPCT can apply filters[1] in the result pipeline to improve the quality of the results, providing the priority list to the user. But the failed test does not have more information than the title and the system and failed logs.

      Acceptance criteria:

      • Provide the user the ability to quickly identify the tests failures related to their environment, read the logs and the test documentation
      • Work with internal teams to understand and, eventually propose a standard to document e2e tests included in openshift/conformance, following the standard from kubernetes/conformance[2]
      • Map opportunities to improve the results, not being limited a code changes (for example hardcoded lists, as it does not scale well)

      Engineering references:

      1) https://redhat-openshift-ecosystem.github.io/provider-certification-tool/dev/#dev-diagram-filters

      2) https://github.com/cncf/k8s-conformance/blob/master/docs/KubeConformance-1.27.md

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              rhn-support-mrbraga Marco Braga
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                PagerDuty