Loading...

XML

Word

Printable

Type: Bug
Resolution: Done-Errata
Priority: Critical
Fix Version/s: rhos-18.0.6
Affects Version/s: rhos-18.0.4
Component/s: edpm-ansible, openstack-ansible-ee
Labels:
None

Story Points:
2
Epic Link:
[Epic]: Updating from 18.0.3 to 18.0.4 fails due to `/var/lib/openstack/config/nova/nova_statedir_ownership.py: no such file or directory`
Blocked:
False
Blocked Reason:

Hide

None

Show
None
Ready:
False
Docs Approval:
Committed
Fixed in Build:
openstack-ansible-ee-container-1.0.7-6
Regression:
None
Release Note Text:

Hide
.Fixed update failure related to nova_statedir_ownership.py

Before this fix, updates from RHOSO18.0.3 to later releases failed with errors related to the missing nova_statedir_ownership.py script.
With this fix, updates from RHOSO 18.0.3 to RHOSO 18.0.6 (Feature Release 2) do not generate these errors.

Show
.Fixed update failure related to nova_statedir_ownership.py Before this fix, updates from RHOSO18.0.3 to later releases failed with errors related to the missing nova_statedir_ownership.py script. With this fix, updates from RHOSO 18.0.3 to RHOSO 18.0.6 (Feature Release 2) do not generate these errors.
Release Note Type:
Bug Fix
Release Note Status:
Done
Intelligence Requested:
Market:
Errata Link:
https://errata.engineering.redhat.com/advisory/146722

Sprint:
Compute FR2 d/t + 18.0.7 BF
sprint_count:
1
Severity:
Important

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

colored textI followed the document[1] in my lab environment but openstack-edpm-update-services deployment failed like the following.

[stack@util ~]$ oc get openstackversion
NAME TARGET VERSION AVAILABLE VERSION DEPLOYED VERSION
openstack-control-plane 18.0.4-20250106.2 18.0.4-20250106.2 18.0.3-20241126.2

$ oc get openstackdataplanedeployment
NAME NODESETS STATUS MESSAGE
data-plane-deploy ["openstack-data-plane"] True Setup complete
edpm-deployment-ipam-ovn-update ["openstack-data-plane"] True Setup complete
edpm-deployment-ipam-update-dataplane-services ["openstack-data-plane"] False Deployment error occurred nodeSet: openstack-data-plane error: backoff limit reached for execution.name update-edpm-deployment-ipam-update-dataplane-services-openstack execution.namespace openstack execution.condition.message: Job has reached the specified backoff limit

and checking the log of the pod,

[stack@util ~]$ oc logs update-edpm-deployment-ipam-update-dataplane-services-opendh2sx -f
...
TASK [osp.edpm.edpm_container_manage : Create containers from /var/lib/openstack/config/containers] ***
included: /usr/share/ansible/collections/ansible_collections/osp/edpm/roles/edpm_container_manage/tasks/create.yml for edpm-compute-0, edpm-compute-1

TASK [osp.edpm.edpm_container_manage : Create containers managed by Podman for /var/lib/openstack/config/containers] ***
[WARNING]: ERROR: Container nova_compute_init exited with code 125 when
createed stderr: time="2025-01-24T03:50:49Z" level=info msg="podman filtering
at log level info" time="2025-01-24T03:50:49Z" level=info msg="Using sqlite as
database backend" time="2025-01-24T03:50:49Z" level=info msg="Not using native
diff for overlay, this may cause degraded performance for building images:
kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled" time="2025-01-24T03:50:49Z"
level=info msg="Setting parallel job count to 13" Error: statfs
/var/lib/openstack/config/nova/nova_statedir_ownership.py: no such file or
directory
fatal: [edpm-compute-0]: FAILED! => {"changed": false, "msg": "Failed containers: nova_compute_init"}
fatal: [edpm-compute-1]: FAILED! => {"changed": false, "msg": "Failed containers: nova_compute_init"}

NO MORE HOSTS LEFT *************************************************************

NO MORE HOSTS LEFT *************************************************************

PLAY RECAP *********************************************************************
edpm-compute-0             : ok=139  changed=11   unreachable=0    failed=1    skipped=22   rescued=0    ignored=0   
edpm-compute-1             : ok=139  changed=11   unreachable=0    failed=1    skipped=22   rescued=0    ignored=0

And, I logged into the compute node and checked /var/log/messages, and I see this ERROR.

Jan 24 04:04:38 edpm-compute-0 python3[54611]: ansible-edpm_container_manage [WARNING] ERROR: Container nova_compute_init exited with code 125 when createed#012stderr: time="2025-01-24T04:04:38Z" level=info msg="podman filtering at log level info"#012time="2025-01-24T04:04:38Z" level=info msg="Using sqlite as database backend"#012time="2025-01-24T04:04:38Z" level=info msg="Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled"#012time="2025-01-24T04:04:38Z" level=info msg="Setting parallel job count to 13"#012Error: statfs /var/lib/openstack/config/nova/nova_statedir_ownership.py: no such file or directoryJan 24 04:04:38 edpm-compute-0 python3[54611]: ansible-edpm_container_manage [WARNING] ERROR: Container nova_compute_init exited with code 125 when createed#012stderr: time="2025-01-24T04:04:38Z" level=info msg="podman filtering at log level info"#012time="2025-01-24T04:04:38Z" level=info msg="Using sqlite as database backend"#012time="2025-01-24T04:04:38Z" level=info msg="Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled"#012time="2025-01-24T04:04:38Z" level=info msg="Setting parallel job count to 13"#012Error: statfs /var/lib/openstack/config/nova/nova_statedir_ownership.py: no such file or directory

I checked if the file (nova_statedir_ownership.py) exists but "No such file or directory"..

[root@edpm-compute-0 cloud-admin]# ls -l /var/lib/openstack/config/nova/nova_statedir_ownership.py
ls: cannot access '/var/lib/openstack/config/nova/nova_statedir_ownership.py': No such file or directory
[root@edpm-compute-0 cloud-admin]# ls -l /var/lib/openstack/config/nova/
total 28
-rw-r--r--. 1 cloud-admin cloud-admin 4819 Jan 16 12:13 01-nova.conf
-rw-r--r--. 1 cloud-admin cloud-admin  170 Jan 16 12:13 02-nova-host-specific.conf
drwxr-xr-t. 2 root        root           6 Jan 16 12:13 ceph
-rw-r--r--. 1 cloud-admin cloud-admin 1919 Jan 16 12:13 config.json
-rw-r--r--. 1 cloud-admin cloud-admin    0 Jan 16 12:13 nova-blank.conf
-rw-r--r--. 1 cloud-admin cloud-admin   72 Jan 16 12:13 ssh-config
-rw-r--r--. 1 cloud-admin cloud-admin  748 Jan 16 12:13 ssh-privatekey
-rw-r--r--. 1 cloud-admin cloud-admin  277 Jan 16 12:13 ssh-publickey
[root@edpm-compute-0 cloud-admin]# find / -name nova_statedir_ownership.py

[1]2.3. Updating the remaining services on the data plane

depends on

OSPRH-14507 Create an integration job that updates from a specific minor version to another specific minor version

Closed

duplicates

OSPRH-13453 RHOSO update 18.0.3 to 18.0.4 fails when updating the remaining services on the data plane

Closed

is cloned by

OSPRH-13726 [Docs] Updating from 18.0.3 to 18.0.4 fails due to `/var/lib/openstack/config/nova/nova_statedir_ownership.py: no such file or directory`

Closed

links to

openstack-k8s-operators/edpm-ansible#869: Apply configuration tasks for Nova on updates

openstack-k8s-operators/edpm-ansible#896: Apply updates for edpm_nova role instead of config

openstack-k8s-operators/install_yamls#1009: [DNM] Apply workaround for OSPRH-13415

RHBA-2025:146722 Release of containers for RHOSO OpenStack EDPM images

mentioned on

Merge request - [DNM] Test U/s adoption job for standalone deployment

Merge request - Updated US source to: 5bf60ec Merge pull request #903 from openshift-cherrypick-robot/cherry-pick-896-to-18.0-fr2

(2 links to, 3 mentioned on)

Assignee:: Bohdan Dobrelia

Reporter:: Masayuki Igawa

Team:: rhos-workloads-compute

Votes:: 0 Vote for this issue

Watchers:: 12 Start watching this issue

Created:: 2025/01/27 1:50 AM

Updated:: 2025/04/16 6:48 PM

Resolved:: 2025/03/20 8:18 AM

Details

Description

Attachments

Issue Links

Easy Agile Planning Poker

Activity

People

Dates

PagerDuty