Summary

All OpenShift-based CI jobs are failing due to an invalid PostgreSQL operator subscription that prevents both PostgreSQL and OpenShift Serverless operators from installing.

Environment

• Affected Branches: release-1.7, release-1.8, main
• OpenShift Catalog: community-operator-index:v4.18
• File: .ibm/pipelines/utils.sh:231

Issue Description

The CI infrastructure attempts to install a PostgreSQL operator package named postgresql from the community-operators catalog, but this package no longer exists in Red Hat's curated catalog index. This causes a cascading failure that blocks ALL operator installations in the openshift-operators namespace, including the serverless-operator.

Current (Broken) Code:

install_crunchy_postgres_ocp_operator(){
  install_subscription postgresql openshift-operators v5 postgresql community-operators openshift-marketplace
  check_operator_status 300 "openshift-operators" "Crunchy Postgres for Kubernetes" "Succeeded"
}

OLM Error:

type: ResolutionFailed
reason: ConstraintsNotSatisfiable
message: 'constraints not satisfiable: no operators found in package postgresql
         in the catalog referenced by subscription postgresql'

Root Cause

The postgresql package was a community distribution of Crunchy Data's PostgreSQL operator that has been deprecated:

• The upstream GitHub repository was archived on July 30, 2021
• Red Hat recently updated their community-operator-index:v4.18 catalog image
• The deprecated postgresql package name was removed from the curated catalog
• The code worked for months because the package existed until this recent catalog update

Historical Context:
PR #2894 (commit fd769706, May 21, 2025) changed the subscription from crunchy-postgres-operator (certified-operators) to postgresql (community-operators). This change worked successfully for several months because the postgresql package still existed in the catalog at that time.

Impact

Blocked CI Jobs (7 job types)

1. OCP Helm Nightly (ocp*helm*nightly)
2. OCP Helm Sealights Nightly (ocp*helm*sealights*nightly)
3. OCP Helm Upgrade Nightly (ocp*helm*upgrade*nightly)
4. OCP Operator Nightly (ocp*operator*nightly)
5. OSD-GCP Helm Nightly (osd-gcp*helm*nightly)
6. OSD-GCP Operator Nightly (osd-gcp*operator*nightly)
7. Pull Request OCP Helm Tests (pull*ocp*helm) - CRITICAL: Blocking all PR merges

Affected Functionality

• External PostgreSQL database testing
• RBAC showcase deployments with external databases
• Orchestrator workflow testing
• Serverless operator installation (blocked by PostgreSQL subscription failure)
• Runtime configuration change tests

Proposed Fix

Revert the subscription to use the certified Crunchy PostgreSQL operator:

install_crunchy_postgres_ocp_operator(){
  install_subscription crunchy-postgres-operator openshift-operators v5 crunchy-postgres-operator certified-operators openshift-marketplace
  check_operator_status 300 "openshift-operators" "Crunchy Postgres for Kubernetes" "Succeeded"
}

Why This Fix Works

• crunchy-postgres-operator package exists in certified-operators catalog
• Currently at version 5.8.5
• Red Hat certified and supported
• This was the original working configuration before May 2025

Branches Requiring Fix

1. main branch - Priority: Critical
2. release-1.7 branch - Priority: Critical
3. release-1.8 branch - Priority: Critical
4. Any active feature branches containing commit fd769706 or 680fac09

Testing Verification

# Verify package exists
oc get packagemanifest crunchy-postgres-operator

# Delete broken subscription
oc delete subscription postgresql -n openshift-operators

# Verify operator installs successfully after fix is applied
oc get csv -n openshift-operators | grep postgres

# Verify serverless operator can now install
oc get csv -n openshift-operators | grep serverless

Additional Information

• Detailed analysis document: POSTGRESQL_OPERATOR_ISSUE.md in repository
• The postgresql and crunchy-postgres-operator packages install the SAME operator, just different package names
• Certified operators from certified-operators catalog should be preferred when available

References

• Breaking change: PR #2894 (commit fd769706)
• Date introduced: May 21, 2025
• Affected file: .ibm/pipelines/utils.sh:231-234