This Jira is to ask for more documentation on the process of replacing Ceph Mons when Openstack is involved.

When replacing Ceph Mons, Openstack does not dynamically update the Ceph Mon IPs on the actively running instances.  As the old Mon IPs are removed, the active instances will fail until their libvirt domains are rebuilt through hard-reboot, live migration, or shelve/ unshelve.

The current "Important" warning of section 11.2 "Removing a Ceph Monitor daemon" [1] reads as if it only applies to director-deployed Ceph that's hyperconverged onto an Openstack controller.   Unfortunately the issue applies to HCI and non-HCI implementations of Ceph when Openstack is involved.

Partner would like this warning in a place more visible in the Openstack and Ceph documentation.

[1]https://docs.redhat.com/en/documentation/red_hat_openstack_platform/17.1/html/installing_and_managing_red_hat_openstack_platform_with_director/assembly_replacing-controller-nodes#proc_removing-a-ceph-monitor-daemon_replacing-controller-nodes

Screenshots

• Attached Image

Bug impact

• Replacing Ceph Mons when Openstack instances are using them could lead to an outage as old mons are removed.

Known workaround

• reuse the old Mon IPs 
• rebuild the libvirt domains of the instances via hard-reboot, live migrate, or shelve/unshelve