Goal:

• 1) Publish the results from CI periodic jobs (aka baseline results) in a persistent storage accessible to the reviewers (persona investigating a provider execution, comparing the report with command 'report --dif')
• 2) Create a interface to navigate through the results
• 3) Get the timers for the executions/jobs (into different variants)
• 4) Publish the summary/report somewhere

Description:

The OPCT periodic jobs executions can provide signals of stability of OPCT executions in specific OCP releases validated by partners. So, as a partner running the validation in their infrastructure in 4.12.1. we will know the expected results when running in know-infrastructure for that specific version.

Information to support each goal:

1) currently the results is published in S3 Bucket in openshift-dev account.

2) the interface to navigate through the results could be AWS Console, S3 service, or a custom HTML page provided to list the objects in the bucket. The problem of the current approach is that the page is listing all objects all the time. It could be improved by indexing the files into static json file, updated when a new result is published. Listing only relevant files, instead of entire bucket.

3) Getting the timers is important to:

• set the expectations of OPCT executions in a given mode (regular, upgrade, disconnected) and OCP version
• measure regressions, for OPCT and OCP
• Seek for improvement in infrastructure allocated for the job

There are different ways to collect the timers, some ideas:

• Crawler and parse the prow job page, getting timers for each step
• Inject control files/vars for each step saving it in the SHARED_DIR and ARTIFACTS_DIR, reading it in the artifacts in GCS. or publishing as a metadata in the persistent storage (mentioned in 1)

4) A report, as described in the task OPCT-46, can be published in the interface described in 2.

Acceptance criteria:

• 1) Persistent storage with an accepted retention policies
• 2) Interface exploring only relevant baseline results
• 3) Get timers from Install time, OPCT execution and results, for each periodic job
• 4) Provide a summarized interface to navigate through the results: a table with each job by row