Feature Overview 

Deployment of separate MariaDB Galera and RabbitMQ clusters for each OpenStack service is desired in some scenarios where the scale or deployment of the OpenStack environment would justify the additional resource usage.

This allows: 

• Impact reduction when a Galera cluster is down.
• Intervention on a focused part of the infrastructure.
• Better distribution of a large number of requests.

Goals

Provide a shared-nothing architecture for Red Hat OpenStack Services on OpenShift (RHOSO) to eliminate performance bottlenecks and single points of failure at scale.

• Create automated testing, including validated architecture enablement.
• Validate the shared-nothing architecture works at an appropriate scale.

Provide documentation to allow the deployment of service-aligned deployments of the database and message bus.

• Create a knowledge base article that broadly captures the general requirements and procedure to implement multiple database and message bus deployments, aligned to each deployed OpenStack service.
• Create official documentation in the OpenStack guide.

Requirements

 
Done - Acceptance Criteria 

A knowledge base article (KBA) is available for consumption by appropriate end users that need access to the deployment in the short term. Any usage of the KBA information in production will likely require a support exception to allow engineering review and considerations.

This feature will be completed when the functionality has been automated, validated, and documented in the OpenStack guide and made generally available for use.

Use Cases - i.e. User Experience & Workflow: 

[Use Case] A partner has created a large scale deployment of RHOSO, with 900 compute nodes and extensive workloads for the first cluster. The partner wishes to achieve 1500 compute nodes for the second RHOSO cluster. In their current Legacy Cloud, the partner has a configuration with one Database cluster per OpenStack service.

[Workflow] The OpenStack administrator will enable the deployment of per-service infrastructure by configuring the RHOSO environment using the standard CustomResource interfaces already provided, along with per-service secrets and other potential per-service objects.

Out of Scope

No additional development should be necessary to implement this feature as it is expected the deployment and configuration will be possible with the current set of CustomResourceDefinitions provided by RHOSO 18.

It may not be possible to transition from a shared architecture to a shared-nothing architecture. Needs validation.

Documentation Considerations

An initial KBA will be provided that gives an example configuration and basic procedures allowing advanced RHOSO administrators to deploy per-service infrastructure (database, message bus). Engineering review will be all that is required to ship this KBA.

A separate mini-content journey that explores a more complete set of documentation will also need to be created and managed as a separate delivery effort targeting a future feature release.

On Upstream side, Large Scale SIG group has already done tests and is recommending  to split the Galera cluster into separate clusters, here [1]

[1]: https://docs.openstack.org/large-scale/journey/configure/database.html

Questions to Answer

• Should this Feature be scoped larger to include automated testing or be limited to a documentation-only feature?
  • if this feature is documentation-only, how should automated testing for this architecture be tracked?
  • could create an Outcome for this overall effort and have separate Features / Initiatives that track the documentation delivery separate of automated testing.
  • YES this feature is now expanded to include the overall delivery of this functionality including automated testing and not just documentation.

Background and Strategic Fit

Some customers already have a shared-nothing architecture deployment. In some cases the scale or architectural decisions for some RHOSO environments justifies the additional resource usage for segregated database and message bus interfaces for each OpenStack service.

Customer Considerations

The customer should be able to follow this procedure as part of their RHOSO deployment planning.

Team Sign Off (Completion while in Planning status)

• All required Epics (known at the time) are linked to the this Feature
• All required Stories, Tasks (known at the time) for the most immediate Epics have been created and estimated
• Add - Reviewers name, Team Name
• Acceptance == Feature as “Ready” - well understood and scope is clear - Acceptance Criteria (scope) is elaborated, well defined, and understood
• Note: Only set FixVersion/s: on a Feature if the delivery team agrees they have the capacity and have committed that capability for that milestone

  Reviewed By	Team Name	Accepted	Notes
  Engineering (dataplane)	rhos-ops-day1day2-edpm
  Engineering (control plane)	rhos-conplat-core-operators
  Engineering (message bus, database)	rhos-ops-platform-services-pidone
  Documentation	rhos-docs
  Product Management	Gil Rosenberg