Uploaded image for project: 'Red Hat Advanced Cluster Management'
  1. Red Hat Advanced Cluster Management
  2. ACM-28010

ACM 2.14.2 and MCE 2.9.2 zStream Test Plan (used to AI tools testing)

XMLWordPrintable

    • ACM 2.15.1, MCE 2.10.1 zStream Test
    • False
    • Hide

      None

      Show
      None
    • False
    • Not Selected
    • To Do
    • 100% To Do, 0% In Progress, 0% Done

      ACM 2.14.2 and MCE 2.9.2 zStream Regression Testing

      Overview

      Comprehensive zStream regression testing for ACM 2.14.2 and MCE 2.9.2 across Regression (Non-UI/UI), Upgrade (Y-stream and Z-stream), Staging, Post-Publishing test paths, and Advisory/Pre-Production validation.

      Environment Configuration

      Environment Matrix Table

      Cluster Type Platform ACM Version OCP Version Purpose
      Hub vSphere 2.14.2 4.20 ACM Hub deployment
      Spoke-1 Hive N/A 4.20 Cluster lifecycle
      Spoke-2 IKS N/A Latest Multi-cloud policy

      Testing Scope

      Overview

      This table defines the testing scope for each component in the zStream release. Components are evaluated for E2E regression testing, Y-stream upgrade testing, and Z-stream upgrade testing based on whether they have bugs requiring retest.

      Automation Test Scope Matrix

      Components QE Focal Fresh Install Y-Upgrade Z-Upgrade Comments
      Application Lifecycle rh-ee-yupchang Yes Yes Yes Full zStream regression testing required
      Cluster Lifecycle rhn-support-dhuynh Yes Yes Yes Full zStream regression testing required
      DR4HUB thuyng Yes No No -
      Discovery rh-ee-msmigiel Yes No No -
      GRC drkho Yes Yes Yes Full zStream regression testing required
      HDRAPP thuyng Yes No No -
      HDRVIRT thuyng Yes No No -
      Installer rh-ee-msmigiel Yes Yes Yes Full zStream regression testing required
      Observability mzardab@redhat.com Yes Yes Yes Full zStream regression testing required
      Search rhn-support-xiyin Yes Yes Yes Full zStream regression testing required
      Server Foundation laisongls Yes Yes Yes Full zStream regression testing required
      VolSync thuyng Yes No No -

      Legend:

      • Fresh Install: Comprehensive fresh installation testing
      • Y-Upgrade: Minor version upgrade testing (e.g., ACM 2.14.x → 2.15.1)
      • Z-Upgrade: Patch version upgrade testing (e.g., ACM 2.15.0 → 2.15.1)
      • QE Focal: Primary QE contact responsible for component testing
      • Comments: Additional testing notes or special requirements

      Testing Logic:

      • ✅ Yes = Testing required for this component
      • ❌ No = Testing not required
      • Default behavior: Components with bugs → Fresh Install, Y-Upgrade, Z-Upgrade all set to "Yes"

      Bug Retest Requirements

      Overview

      This zStream release includes critical bug fixes that require comprehensive retesting across multiple components and squads. Each bug must be retested to verify the fix is working as expected in the target release environment.

      Bug Organization by Component

      Application Lifecycle (3 bugs)

      QE Focal: rh-ee-yupchang

      Squad: @acm-qe-workload-mgmt

      Jira Query: View all Application Lifecycle bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-25668 ACM 2.14.2 Major Review [ACM 2.14] Creating App Subscription with time window causes application manager pod crashes
      ACM-25543 ACM 2.14.2 Normal Closed AppSub Helm versioning is incompatible
      ACM-21272 ACM 2.14.2 Undefined Review Cannot edit AppSet ClusterSet [release-2.14]

      Business Continuity (1 bugs)

      QE Focal: Unassigned

      Squad: Unassigned

      Jira Query: View all Business Continuity bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-26062 ACM 2.14.2 Major Review [release-2.14] When restoring acm using backup names, credentials backup is not picked up

      Cluster Lifecycle (1 bugs)

      QE Focal: rhn-support-dhuynh

      Squad: @acm-qe-workload-mgmt

      Jira Query: View all Cluster Lifecycle bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-25074 MCE 2.9.2 Blocker Closed [MCE 2.9] Rebuild image: cluster-image-set-controller-rhel9

      Console (7 bugs)

      QE Focal: rhn-support-dhuynh

      Squad: @acm-qe-workload-mgmt

      Jira Query: View all Console bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-26082 MCE 2.9.2 Critical Closed [mce-29] console-mce-mce-29 - Konflux compliance failure
      ACM-24961 MCE 2.9.2 Critical Review Unable to set the attachDefaultNetwork: false while installing HCP cluster using GUI
      ACM-27468 MCE 2.9.2 Major Review console-mce-console readiness and liveness probe failures at scale during ACM ZTP with AAP and EDA [MCE 2.9]
      ACM-25039 ACM 2.14.2 Major Closed console-mce pod CrashLoopBackOff due to probe failure - improve performance [release-2.14]
      ACM-25583 MCE 2.9.2 Normal Review [MCE UI] - (2.14.z) Cluster pool records misaligned
      ACM-24883 ACM 2.15.0 Normal Closed Insights-client error; cluster recommendations are empty
      ACM-21272 ACM 2.14.2 Undefined Review Cannot edit AppSet ClusterSet [release-2.14]

      GRC (2 bugs)

      QE Focal: drkho

      Squad: @acm-qe-grc

      Jira Query: View all GRC bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-14842 ACM 2.15.0 Critical Closed Failed to retrieve Job names when using unified platform url(AAP 2.5) in the RHAAP credentials
      ACM-25543 ACM 2.14.2 Normal Closed AppSub Helm versioning is incompatible

      Global Hub (1 bugs)

      QE Focal: Unassigned

      Squad: Unassigned

      Jira Query: View all Global Hub bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-23840 ACM 2.14.2 Minor Closed registration-controller always restart when do cluster migration

      Hive (1 bugs)

      QE Focal: Unassigned

      Squad: Unassigned

      Jira Query: View all Hive bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-26219 ACM 2.14.2 Blocker New Managed cluster fails to deploy: clusterdeployments.hive.openshift.io stuck in status "provisioning"

      HyperShift (1 bugs)

      QE Focal: rhn-support-dhuynh

      Squad: @acm-qe-workload-mgmt

      Jira Query: View all HyperShift bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-28134 MCE 2.9.2 Critical Closed [mce-29] hypershift-cli-mce-29 - Konflux compliance failure

      Image Based Install Operator (1 bugs)

      QE Focal: Unassigned

      Squad: Unassigned

      Jira Query: View all Image Based Install Operator bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-26389 ACM 2.15.0 Blocker Closed IBIO - Update go.mod Hive dependency

      Infrastructure Operator (1 bugs)

      QE Focal: Unassigned

      Squad: Unassigned

      Jira Query: View all Infrastructure Operator bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-26034 ACM 2.14.z Normal Resolved [ACM 2.14] BareMetalHost CR stuck in deleting phase after ZTP removal in case of failed installation

      Installer (3 bugs)

      QE Focal: rh-ee-msmigiel

      Squad: @acm-qe-install

      Jira Query: View all Installer bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-26334 ACM 2.14.2 Critical Closed site config failed to update resource during ACM upgrading
      ACM-26083 MCE 2.9.2 Critical Closed [mce-29] discovery-operator-mce-29 - Konflux compliance failure
      ACM-24834 ACM 2.14.2 Normal Closed upgrading ACM 2.13.4 to 2.14.0 cause siteConfig pod container kube-rbac-proxy to get in error

      Multicluster Networking (1 bugs)

      QE Focal: Unassigned

      Squad: Unassigned

      Jira Query: View all Multicluster Networking bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-24797 ACM 2.14.2 Major Testing Connection status between two clusters goes from Connected to Connecting from both directions

      Observability (4 bugs)

      QE Focal: mzardab@redhat.com

      Squad: @observatorium-support

      Jira Query: View all Observability bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-27834 ACM 2.14.2 Major Review Observability tries to determine if a cluster is OCP too early in deployment
      ACM-26168 ACM 2.14.2 Minor Closed Missing CRD status never recovers when disabling MCOA
      ACM-21724 ACM 2.15.0 Normal Closed Hub Info Secret does not get updated when custom ingress cert is updated
      ACM-27633 ACM 2.14.2 Undefined Review [2.14] observability=disabled label fails to remove all observability addon resources

      Search (3 bugs)

      QE Focal: rhn-support-xiyin

      Squad: @acm-qe-search

      Jira Query: View all Search bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-28604 ACM 2.14.2 Major Closed Search cluster scoped resource rbac check [2.14]
      ACM-23280 ACM 2.15.0 Major Closed Search-postgres unable to configure shared memory size
      ACM-24883 ACM 2.15.0 Normal Closed Insights-client error; cluster recommendations are empty

      Server Foundation (5 bugs)

      QE Focal: laisongls

      Squad: @acm-qe-serverfoundation

      Jira Query: View all Server Foundation bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-26081 MCE 2.9.2 Critical Closed [mce-29] managed-serviceaccount-mce-29 - Konflux compliance failure
      ACM-26047 MCE 2.9.2 Critical Closed [mce-29] managed-serviceaccount-mce-29 - Konflux compliance failure
      ACM-26045 MCE 2.9.2 Critical Closed [mce-29] cluster-proxy-mce-29 - Konflux compliance failure
      ACM-23686 MCE 2.9.2 Major Review klusterlet remained on ROSA HCP provisioned by CAPA after detaching
      ACM-23840 ACM 2.14.2 Minor Closed registration-controller always restart when do cluster migration

      SiteConfig Operator (4 bugs)

      QE Focal: treywest96

      Squad: @trwest@redhat.com

      Jira Query: View all SiteConfig Operator bugs

      Jira Issue Fix Version Priority Status Summary
      ACM-26057 ACM 2.15.0 Critical Resolved IBI install failing due to NTP servers in seed not getting updated
      ACM-26969 ACM 2.14.2 Major Closed ClusterInstance detach via GitOps leaves observability addons on managedcluster
      ACM-25222 ACM 2.14.2 Major Testing scale-in failed: admission webhook "clusterinstances.siteconfig.open-cluster-management.io" denied the request: detected unauthorized changes in immutable fields
      ACM-23880 ACM 2.15.0 Normal Closed Default IBI templates in SiteConfig operator do not support BareMetalHosts in separate namespaces via ClusterInstance.spec.nodes.hostRef

      Total Bugs: 36 (ACM: 24, MCE: 12)

      Test Paths

      1. Regression Testing

      1.1 ACM 2.14.2 Regression (Non-UI, FIPS Enabled)

      Test Configuration:

      ACM Version OCP Version Hub Platform Spoke-1 (Hive) Spoke-2 (IKS) FIPS Mode Test Type
      2.14.2 4.20 vSphere AWS / OCP 4.20 IKS / Latest Enabled Non-UI Only

      Jenkins Pipeline: e2e_non-ui_test_pipeline

      Note: UI testing is NOT included in zStream regression testing.

      2. Upgrade Testing

      2.1 MCE Y-Stream Upgrade (FIPS Disabled)

      Test Configuration:

      Upgrade Path OCP Version Hub Platform Spoke-1 (Hive) Spoke-2 (IKS) FIPS Mode
      MCE 2.8.x → 2.9.2 4.20 vSphere AWS / OCP 4.20 IKS / Latest Disabled

      Jenkins Pipeline: e2e_upgrade_pipeline

      2.2 ACM Y-Stream Upgrade (FIPS Disabled)

      Test Configuration:

      Upgrade Path OCP Version Hub Platform Spoke-1 (Hive) Spoke-2 (IKS) FIPS Mode
      ACM 2.13.x → 2.14.2 4.20 vSphere AWS / OCP 4.20 IKS / Latest Disabled

      Jenkins Pipeline: e2e_upgrade_pipeline

      2.3 ACM Z-Stream Upgrade (FIPS Disabled)

      Test Configuration:

      Upgrade Path OCP Version Hub Platform Spoke-1 (Hive) Spoke-2 (IKS) FIPS Mode
      ACM 2.14.1 → 2.14.2 4.20 vSphere AWS / OCP 4.20 IKS / Latest Disabled

      Jenkins Pipeline: e2e_upgrade_pipeline

      3. Staging Testing

      3.1 ACM 2.14.2 Staging Testing

      Test Configuration:

      ACM Version OCP Version Hub Platform Spoke-1 (Hive) Spoke-2 (IKS) FIPS Mode Environment
      2.14.2 (Staging) 4.20 vSphere AWS / OCP 4.20 IKS / Latest Disabled Staging

      3.2 MCE 2.9.2 Staging Testing

      Test Configuration:

      MCE Version OCP Version Hub Platform Spoke-1 (Hive) Spoke-2 (IKS) FIPS Mode Environment
      2.9.2 (Staging) 4.20 vSphere AWS / OCP 4.20 IKS / Latest Disabled Staging

      4. Post-Publishing Testing

      4.1 ACM 2.14.2 Post-Publishing Testing

      Test Configuration:

      ACM Version OCP Version Hub Platform Spoke-1 (Hive) Spoke-2 (IKS) FIPS Mode Environment Catalog Source
      2.14.2 (Published) 4.20 vSphere AWS / OCP 4.20 IKS / Latest Disabled Production-like Published OperatorHub

      4.2 MCE 2.9.2 Post-Publishing Testing

      Test Configuration:

      MCE Version OCP Version Hub Platform Spoke-1 (Hive) Spoke-2 (IKS) FIPS Mode Environment Catalog Source
      2.9.2 (Published) 4.20 vSphere AWS / OCP 4.20 IKS / Latest Disabled Production-like Published OperatorHub

      5. Advisory and Pre-Production Testing

      5.1 Prepare and Review Advisory

      5.2 Pre-Production Pushing Testing

      Test Execution Workflow

      1. Identify and organize bugs for retest - Retrieve all bugs from ACM 2.14.2 and MCE 2.9.2 releases
      2. Create squad-specific retest tasks - Use automated tooling to create individual retest tasks for each squad
      3. Deploy test environment - Deploy ACM 2.14.2 on vSphere hub (OCP 4.20)
      4. Provision spoke clusters - Provision spoke clusters (Hive + IKS)
      5. Execute bug retests - Squads verify bug fixes
      6. Execute regression tests - Run regression test pipelines
      7. Execute upgrade tests - Run upgrade test pipelines
      8. Execute staging tests - Run staging test suite
      9. Execute post-publishing tests - Run post-publishing test suite
      10. Prepare and review advisory - Prepare advisory content and documentation
      11. Execute pre-production testing - Validate pre-production deployment
      12. Document and report results - Document all test results and file issues for failures

              huichen@redhat.com Hui Chen
              huichen@redhat.com Hui Chen
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: