Uploaded image for project: 'OPCT - OpenShift Provider Compatibility Tool'
  1. OPCT - OpenShift Provider Compatibility Tool
  2. OPCT-46 [CI] Create summarized report of results in CI
  3. OPCT-203

[CI] collect insights from the CI periodic jobs

    XMLWordPrintable

Details

    • Sub-task
    • Resolution: Unresolved
    • Normal
    • None
    • None
    • None
    • False
    • Hide

      None

      Show
      None
    • False

    Description

      Extract metrics from CI executions to further, and quickly, be analysed to detect anomalies (OPCT-46)

      Metrics we expected on the results for each execution:

      • Total counters[1] for each result by plugin: Total/Passed/Skipped/Failed[...]
      • Failure ratio (counter of the operation failed/total) by plugin
      • Failure ratio after applying filters (recently added 'report' command)
      • Total time to create the cluster*, run the tool, destroy the cluster*

      ENGINEERING DETAILS

      [1] Previous work / examples:

      https://github.com/redhat-openshift-ecosystem/provider-certification-tool/pull/37#issuecomment-1372704688

      https://github.com/redhat-openshift-ecosystem/provider-certification-tool/pull/38#issuecomment-1372703547

      [2] WIP automation to automatically create AWS cluster [agnostic], run OPCT, and destroy:  https://github.com/redhat-openshift-ecosystem/provider-certification-tool/pull/38

      Attachments

        Issue Links

          blocks

          Spike - Represents a research-related task. OPCT-46 [CI] Create summarized report of results in CI

          • Normal - To be worked after urgent and high priorities are resolved. This term is chosen when the severity of an issue is relatively close to the level of effort to fix it. The existence of an easily implemented workaround can also lead to this priority level instead of a higher priority.
          • To Do
          relates to

          Task - Represents a small unit of work that is not end-user facing. OPCT-178 [CI] Create CI periodic jobs for v0.3 (upgrade)

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed

          Activity

            People

              Unassigned Unassigned
              rhn-support-mrbraga Marco Braga
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                PagerDuty