WHY
All e2e tests should be passing on RHOAM installations

WHAT
A few e2e tests are failing on fresh installs from the OBO feature branch. We aren't able to run these tests via prow checks since prow runs on OCP clusters and are instead using a nightly pipeline to run them on OSD clusters. The pipeline can be found here and these are the broken tests:

• C03 - Verify that alerting mechanism works
• C04 - Verify Alerts exist - See followup ticket MGDAPI-5884
• Verify prometheus metrics scrapped
• Validate resource requirements are set
  • Due to the number of new pods OBO has introduced, it's probably best to just add a LimitRange CR to enforce default request values for all Pods in the "redhat-rhoam-operator-observability" Namespace. This is what we are doing for the other RHOAM namespaces. The LimitRange CR should be added via MTBR's PnP mechanism but we will need to create a new phase for it. This is because the LimitRange CR will only act on Pods that are created after it exists and the only way to guarentee the order of resource creation with the package-operator is using seperate phases. Therefore we should create a new phase called "limitrange" that goes after the "namespace" phase but before the "observability" phase.

HOW
Spin up an OSD cluster, checkout the OBO feature branch and install RHOAM using a CatalogSource. Then run the e2e tests one at a time to debug and fix them.

TESTS

• C03 - Verify that alerting mechanism works - Link
• C04 - Verify Alerts exist - Link
• Verify prometheus metrics scrapped - Link
• Validate resource requirements are set - Link

DONE

• All the affected e2e tests are now passing