-
Epic
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
None
-
As an developer, I care about the order of the nova controller services upgrade during 18.0 minor update
-
False
-
-
False
-
Not Selected
-
Proposed
-
Proposed
-
To Do
-
Proposed
-
Proposed
-
-
Problem description:
During a minor update in 18.0 Nova controller services updated in parallel. The nova-operator updating the StatefulSet of the services with the new container image without any ordering and k8s implementing a rolling upgrade across the replicas of the same StatefulSet (openstack nova service). This means that:
- new nova-api services might be available before all the nova-conductor services are updated to the new version
- different versions of nova-conductor running at the same time during the rolling update of the conductor StatefulSet
This could cause upgrade problems. For example a new nova-api service could send a new object version to an old nova-compute service. Then that compute service asks the nova-conductor to backlevel the object, but if the conductor is also old it would not know how to do it.
This is not a problem during a minor update across upstream point releases of an already GAd nova version. As upstream we are not backporting new object versions to stable branches. However downstream we do backport features to FR releases potentially containing new object version. So downstream we need a stricter minor update sequence.
Goal:
Sequence the minor update of nova services so that the nova-conductor services (both the super and the cell conductors) are always fully updated before nova-api is started to be updated.
Acceptance Criteria:
- Test case exists that shows that all the nova-conductor services are running with the new container image before nova-api container images changed.
- A Note exists in the 18.0 minor update doc to emphasize that the nova-compute services on the EDPM node should only be updated after the full control plane update is finished. (I.e. be careful when updating the OVN services on the EDPM node not to include the nova service to that step)