Background

For galaxy.ansible.com we historically have had downtime for our upgrades. We have not done regular upgrades to galaxy.ansible.com for ~2 years, and therefore have not had regular downtime. Note: The average traffic per minute has likely quadrupled in these ~2 years.

There is a desire to change so that our upgrades will not include downtimes. 

Usually the only reason for downtime is to apply migrations. There is an in-progress process change in pulp to start writing migrations in a way that can be run without downtime. The galaxy team will need to adopt this as well. However there will be times, for example due to new feature work, that we will need to bring the system down for migrations. We may want to investigate how often this will happen based on the adoption of the process.

One way to minimize disruption is to do A B deployments. An example: run 2 instances of the website prior to upgrade, have galaxy.ansible.com point to a read-only instance while we upgrade the other instance. Then afterwards point galaxy.ansible.com to the upgraded instance.