Description

Octavia scenario tests (run via uni-beta job and tobiko) failed because the Amphora image was deleted. This Amphora image is a critical part of the OpenStack Control Plane deployment ("official" resource created during RHOSO installation) and is not a temporary testing resource.
In addition, horizontests should not delete other Octavia resources created during the RHOSO deployment, such as Octavia mgmt networks and routers.

Root Cause

Investigation pinpointed the Horizon cleanup script (clean_leftover_images
in run_horizontest.sh) [1], which runs

openstack image delete '{}' 

on all images in the environment, deleting the Amphora image. This occurs because Horizon tests run last in the job and are designed to clean up "garbage" from prior tests (like Tobiko/Tempest).

Test Failure Details

• Failing Action: Attempting to create a Listener.

• Error in Octavia-worker log:

octavia.common.exceptions.ComputeBuildException: Failed to build compute instance due to: Failed to retrieve image with amphora-image tag.

• Deletion in Glance: A deletion entry was found in the Glance log for the Amphora image's UUID:

DELETE /v2/images/c5701174-8ed1-41d7-b546-e86f76be47b6 

Temporary Workaround

To unblock the tests, the PR [2] order was changed so that Octavia tests run before Horizon tests. It was also recommended to add the parameter cifmw_test_operator_tobiko_cleanup: true to the Tobiko MR to ensure the cleanup of temporary Tobiko resources (like LBs) to avoid impacting Horizon.

Requirement

The Horizon cleanup script must be corrected to exclude system-critical resources, specifically the Amphora image, or any image that is not explicitly created by Horizon tests. The image is part of the deployment and should not be considered "garbage."

Suggested Fix / Action Items

1. Update the Horizon cleanup script (clean_leftover_images in run_horizontest.sh) to filter and exclude the Amphora image (e.g., by name, tag, or status).
2. Review whether the cleanup script should be deleting resources outside of the projects created by the Horizon tests.

[1] - https://github.com/openstack-k8s-operators/tcib/blob/main/container-images/tcib/base/os/horizontest/run_horizontest.sh#L55-L58
[2] - https://gitlab.cee.redhat.com/ci-framework/ci-framework-jobs/-/merge_requests/1748/diffs