Epic Goal

Validate the catalogs (excluding content) by creating a set of tests for each OCP minor version that QE can use. These tests aim to identify inconsistencies and patterns across catalogs.

Why is this important?

• Standardized catalogs improve reliability and usability.
• Enable early the detection of issues with defined validation tests.
• Consistency avoids compatibility issues across catalogs.
• Identifies inconsistencies

Following some examples of scenarios

• Catalog structure validation
  • Compare JSON vs. Pogreb cache at appropriate breakpoints
  • Validate embedded binary vs. bin-less vs. bin + extractContent approaches
  • Ensure correct bundle metadata format (e.g., OCP 4.17+ requires olm.csv.metadata)
  • Define scope: Should validation apply only to channel heads or all bundles?

• Consistency in metadata and tagging
  • Check image tags/labels for standardization
  • Ensure consistent channel formatting across catalogs
  • Verify whether catalogs use standardized channel names (candidate/fast/stable) or other variations
  • Validate semver bundle versions

• Catalog evolution and schema enforcement
  • Ensure a consistent pattern for channel construction (e.g., use of skipRange, replaces, or only skips)
  • Assess whether IIB enforces opinions that impact all input catalogs (e.g., binary-less catalog output: does it benefit ISV/community/Konflux, or only select catalogs?)
  • Address FBC flexibility: Should we enforce that an olm.bundle is present for all entries referenced in graph relationships? This could be a prerequisite for migrating existing schemas to next-version formats.

Acceptance Criteria

• Design of checks defined
• Implementation of a set of checks running automatically