Goal:

As a Cert-Ops review, I would like to have an clear Check/SLO failing if the OpenShift installation is older than expected to prevent false-positive errors while reviewing the results, so I can have more confidence of checks and failed tests.

Description:

OPCT states in the documentation[1] that a fresh OpenShift installation, new cluster, must be created before running the conformance tests to prevent false positives.

This requirement comes from expected behaviors in OpenShift CI.

To check if the cluster is old, the following steps must be performed:

• Extract the timestamp the cluster has been installed from cluster version
• check when the OPCT server has finished
• if the delta is older than 10 hours (double of expected OPCT execution time), the check/SLO must fail

Checking manully:

• Time the cluster was installed

$ mkdir results
$ tar xfz opct_202408200704_96b783d7-2030-412a-853a-e16fd2d8644a.tar.gz -C results/

$ omc-devel use results/plugins/99-openshift-artifacts-collector/results/global/artifacts_must-gather.tar.xz 

$ omc get clusterversion version -o json | jq .status.history[].completionTime
"2024-08-15T05:59:53Z" 

• Time the published conformance test has finished:

$ opct report --log-level=debug --skip-server --save-to report/ opct_202408200704_96b783d7-2030-412a-853a-e16fd2d8644a.tar.gz  

$ jq '.provider.runtime.serverLogs[] | select(.name=="server finished").time ' report/opct-report.json 
"2024-08-20T09:36:38Z"

The delta above is more than 5 days, which trigger the failed check/SLO.

REFERENCES

Slack thread: https://redhat-internal.slack.com/archives/C03UZKBRHT5/p1725286162156489?thread_ts=1724848202.361499&cid=C03UZKBRHT5 

Partner impacted: https://issues.redhat.com/browse/OPCT-303