-
Bug
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
None
rhoso18 environment.
From nova-api log:
2026-03-10 16:26:31.244 11 DEBUG nova.servicegroup.drivers.db [None req-e29ba2da-ea56-4faf-b3b8-515ad35660c2 3e718b86717142ccbb82f5a96c57db13 faac9d4b644844408ff6cc7178e54a3f - - default default] Seems service nova-compute on host nova-compute01 is down. Last heartbeat was 2026-02-19 20:47:17. Elapsed time is 1625954.244931 is_up /usr/lib/python3.9/site-packages/nova/servicegroup/drivers/db.py:76
This issue occurred when trying to do a volume snapshot of an in-use volume.
nova-compute services generally worked fine; a VM was created with volume attachments just prior to this failure.
However, nova-compute service is up. The service status seems to have come from an old soft-deleted service record in the db.
sh-5.1$ openstack compute service list |grep nova-compute
| 6357ba69-8961-46a8-b34f-fb88f80afe63 | nova-compute | nova-compute01 | nova | enabled | up | 2026-03-09T16:14:24.000000 |
| 5e13cc19-b6d9-45ff-b6a8-3df0bc595282 | nova-compute | nova-compute00 | nova | enabled | up | 2026-03-09T16:14:24.000000 |
select updated_at, deleted_at, deleted, host, `binary`, last_seen_up from services where host = 'nova-compute01';
--------------------------------------------------------------------------------------------------------------+
| updated_at | deleted_at | deleted | host | binary | last_seen_up |
--------------------------------------------------------------------------------------------------------------+
| 2026-02-19 20:47:17 | 2026-02-20 19:13:06 | 3 | nova-compute01 | nova-compute | 2026-02-19 20:47:17 |
| 2026-02-23 17:25:23 | 2026-02-23 20:18:10 | 4 | nova-compute01 | nova-compute | 2026-02-23 17:25:23 |
| 2026-02-24 17:17:37 | 2026-02-24 17:17:41 | 6 | nova-compute01 | nova-compute | 2026-02-24 17:17:37 |
| 2026-02-24 17:25:58 | 2026-02-24 17:26:06 | 8 | nova-compute01 | nova-compute | 2026-02-24 17:25:58 |
| 2026-02-24 17:30:20 | 2026-02-24 17:30:23 | 10 | nova-compute01 | nova-compute | 2026-02-24 17:30:20 |
| NULL | 2026-02-24 19:38:47 | 12 | nova-compute01 | nova-compute | NULL |
| 2026-03-06 16:56:12 | 2026-03-06 19:37:08 | 14 | nova-compute01 | nova-compute | 2026-03-06 16:56:12 |
| 2026-03-10 17:00:48 | NULL | 0 | nova-compute01 | nova-compute | 2026-03-10 17:00:48 |
--------------------------------------------------------------------------------------------------------------+
When the 'deleted' service records were purged the above operation completed successfully.
delete from services where deleted > 0;
Seems this is_up operations should exclude deleted services.
