Feature Overview

This feature selects applicable quality and stability metrics from VIRTSTRAT-486 for OpenShift 4.22.

Goals

The goal of this feature is to aim to consistently deliver high-quality, stable, and reliable product releases by focusing on addressing issues and setting standards for testing.

• End-users, and the product and engineering teams benefit from this feature because metrics have been quantified to be more specific and achievable, while maintaining flexibility and independence to accommodate for differing standards between teams
• This feature allows us to better understand where we can improve in product quality and stability

Requirements

Monitor and Achieve Product Quality Index

1. Number of blocking/critical/regression issues is 0.
2. Major/critical/blocker bugs arrival rate has decreased with convergence to 0 prior to GA.
3. The total unresolved Bugs Backlog is decreasing release over release.

Monitor and Achieve Test Stability

1. 99% of the regression tests are passing
2. 90% of the failures are classified as Product Bug/Automation issue/System Issue

Monitor and Achieve Code and Test Coverage

1. Automation test coverage should grow release by release accordingly with pre-established metrics

Bug  Escapes - Close loop

1. 80% of Bugs escapes completed via VME Close Loop RCA process
2. Areas of improvement are recognized and improved upon release by release.