Description:

We recently integrated the Default Catalog Tests with Sippy. These tests have been consistently passing (100% success rate). However, due to intermittent issues with the OCP infrastructure, we have been receiving false alarms, which introduce unnecessary noise and increase the risk of overlooking real failures.

See, i.e.: https://sippy.dptools.openshift.org/sippy-ng/jobs/4.18?filters=%7B%22items%22%3A%5B%7B%22columnField%22%3A%22name%22%2C%22operatorValue%22%3A%22equals%22%2C%22value%22%3A%22periodic-ci-openshift-operator-framework-operator-controller-release-4.18-periodics-default-catalog-consistency%22%7D%5D%7D

Background

• Default Catalog Tests run every 4 hours.

• In the event of a real issue, we would observe consistent failures.

• The current threshold is 90%, which is too sensitive given the infra noise.

• Most false alarms are not due to test logic, but OCP infrastructure instabilities.

Request

Lower the alert threshold in Sippy for the Default Catalogue Tests from 90% to 60%.

Rationale

• A 60% threshold will still surface real issues, as real failures would cause repeated, visible drops below that line.

• This adjustment will significantly reduce false positives, helping us maintain trust in alerting signals and avoid alert fatigue.