-
Bug
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
None
-
-
False
-
?
-
rhos-workloads-compute
-
None
-
-
-
-
-
Important
To Reproduce Steps to reproduce the behavior:
- Deploy an overcloud, 16.2 or 17.1
- Nova has the following database management jobs have been created by default in nova_api_cron:
[root@brq-ctrl01 ~]# podman exec nova_api_cron tail -6 /var/spool/cron/nova # Puppet Name: nova-manage db archive_deleted_rows PATH=/bin:/usr/bin:/usr/sbin SHELL=/bin/sh 1 0 * * * sleep `expr ${RANDOM} \% 3600`; nova-manage db archive_deleted_rows --max_rows 1000 --before `date --date='today - 90 days' +\%F` --until-complete --all-cells >>/var/log/nova/nova-rowsflush.log 2>&1 # Puppet Name: nova-manage db purge PATH=/bin:/usr/bin:/usr/sbin SHELL=/bin/sh 0 5 * * * sleep `expr ${RANDOM} \% 3600`; nova-manage db purge --before `date --date='today - 14 days' +\%D` --all-cells >>/var/log/nova/nova-rowspurge.log 2>&1 [root@brq-ctrl01 ~]#
- Cinder has the following database management jobs have been created by default in cinder_api_cron:
[root@brq-ctrl01 ~]# podman exec cinder_api_cron tail -1 /var/spool/cron/cinder 1 0 * * * sleep `expr ${RANDOM} \% 3600`; cinder-manage db purge 30 >>/var/log/cinder/cinder-rowsflush.log 2>&1 [root@brq-ctrl01 ~]#
- None of the output files have anything in their logs, nor have them been rotated:
[root@brq-ctrl01 ~]# ls -ltr /var/log/containers/{nova/nova-manage.log*,nova/nova-rows*,cinder/cinder-rowsflush.log*} -rw-r--r--. 1 42407 42407 0 Jul 8 2023 /var/log/containers/cinder/cinder-rowsflush.log -rw-r--r--. 1 42436 42436 0 Jul 8 2023 /var/log/containers/nova/nova-rowspurge.log -rw-r--r--. 1 42436 42436 0 May 2 2024 /var/log/containers/nova/nova-manage.log -rw-r--r--. 1 42436 42436 0 May 2 2024 /var/log/containers/nova/nova-rowsflush.log [root@brq-ctrl01 ~]#
Expected behavior
- Jobs run and log output
Bug impact
- This issue was found while troubleshooting errors in a customer upgrade, and is (allegedly) cause of database performance issues.
Known workaround
- Manual run of the jobs complete successfully
Additional context
- See customer case attached for extended details
