In our CI/CD, we use the following command to determine if a specific OCP/RHCOS version is available: "wget --spider https://mirror.openshift.com/pub/openshift-v4/s390x/dependencies/rhcos/4.14/4.14.0/rhcos-4.14.0-s390x-live-rootfs.s390x.img". And it used to be working since OCP 4.11. Recently (July 2024), using wget --spider will resolve a 403 http error with a broken link message, prior to that the server would send 2XX if the version exists. According to wget document, using --spider option will make wget send a HEAD instead of GET request, and I assume prior to the change, the mirror server will respond 2XX status code to both HEAD and GET requests. Now, the server will respond with 403 even if the version exists using --spider option, and if user only use wget (without --spider) option, wget will download the binary as usual.

    test against any existing OCP/RHCOS binary using <wget> with <--spider> option. e.g. "wget --spider https://mirror.openshift.com/pub/openshift-v4/s390x/dependencies/rhcos/4.14/4.14.0/rhcos-4.14.0-s390x-live-rootfs.s390x.img"

    1. In terminal
    2. Issue "wget --spider https://mirror.openshift.com/pub/openshift-v4/s390x/dependencies/rhcos/4.14/4.14.0/rhcos-4.14.0-s390x-live-rootfs.s390x.img"
    3. You should see
wget --spider --method=GET https://mirror.openshift.com/pub/openshift-v4/s390x/dependencies/rhcos/4.14/4.14.0/rhcos-4.14.0-s390x-live-rootfs.s390x.img
Spider mode enabled. Check if remote file exists.
...
HTTP request sent, awaiting response... 403 Forbidden
Remote file does not exist -- broken link!!!     

    wget --spider --method=GET https://mirror.openshift.com/pub/openshift-v4/s390x/dependencies/rhcos/4.14/4.14.0/rhcos-4.14.0-s390x-live-rootfs.s390x.img
Spider mode enabled. Check if remote file exists.
--2024-07-18 16:13:24--  https://mirror.openshift.com/pub/openshift-v4/s390x/dependencies/rhcos/4.14/4.14.0/rhcos-4.14.0-s390x-live-rootfs.s390x.img
Resolving mirror.openshift.com (mirror.openshift.com)... 13.226.34.31, 13.226.34.53, 13.226.34.89, ...
Connecting to mirror.openshift.com (mirror.openshift.com)|13.226.34.31|:443... connected.
HTTP request sent, awaiting response... 307 S3Redirect
Location: https://2209d6dda25891dee55b086a469130de.r2.cloudflarestorage.com/art-srv-enterprise/pub/openshift-v4/s390x/dependencies/rhcos/4.14/4.14.0/rhcos-4.14.0-s390x-live-rootfs.s390x.img?response-content-disposition=attachment%3B%20filename%3D%22rhcos-4.14.0-s390x-live-rootfs.s390x.img%22&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=36cc88a3d563d343cfdd69cbe5af624a%2F20240718%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240718T201324Z&X-Amz-Expires=1200&X-Amz-SignedHeaders=host&X-Amz-Signature=5c3b44ad8f04efb4fc1f32175fcc161f283c51f7f578a750c2d6d2204712dcf8 [following]
Spider mode enabled. Check if remote file exists.
--2024-07-18 16:13:24--  https://2209d6dda25891dee55b086a469130de.r2.cloudflarestorage.com/art-srv-enterprise/pub/openshift-v4/s390x/dependencies/rhcos/4.14/4.14.0/rhcos-4.14.0-s390x-live-rootfs.s390x.img?response-content-disposition=attachment%3B%20filename%3D%22rhcos-4.14.0-s390x-live-rootfs.s390x.img%22&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=36cc88a3d563d343cfdd69cbe5af624a%2F20240718%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240718T201324Z&X-Amz-Expires=1200&X-Amz-SignedHeaders=host&X-Amz-Signature=5c3b44ad8f04efb4fc1f32175fcc161f283c51f7f578a750c2d6d2204712dcf8
Resolving 2209d6dda25891dee55b086a469130de.r2.cloudflarestorage.com (2209d6dda25891dee55b086a469130de.r2.cloudflarestorage.com)... 104.18.9.90, 104.18.8.90
Connecting to 2209d6dda25891dee55b086a469130de.r2.cloudflarestorage.com (2209d6dda25891dee55b086a469130de.r2.cloudflarestorage.com)|104.18.9.90|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
Remote file does not exist -- broken link!!!

    wget --spider https://google.com                130 ↵ ──(Fri,Jul19)─┘
Spider mode enabled. Check if remote file exists.
--2024-07-19 09:44:22--  https://google.com/
Resolving google.com (google.com)... 172.217.165.142
Connecting to google.com (google.com)|172.217.165.142|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://www.google.com/ [following]
Spider mode enabled. Check if remote file exists.
--2024-07-19 09:44:22--  https://www.google.com/
Resolving www.google.com (www.google.com)... 142.250.80.68
Connecting to www.google.com (www.google.com)|142.250.80.68|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Remote file exists and could contain further links,