Epic Goal

ACM/MCE web-console updates should send clusters to by-digest pullspecs like quay.io/openshift-release-dev/ocp-release@sha256:05ba8e63f8a76e568afe87f182334504a01d47342b6ad5b4c3ff83a2463018bd instead of by-tag pullspecs like quay.io/openshift-release-dev/ocp-release:4.14.1-x86_64.

Why is this important?

As described in oc#390, by-tag pullspecs expose the consuming cluster to registry behavior like:

1. You ask to update to a by-tag pullspec.
2. Cluster updates.
3. Someone clobbers the tag you used in the registry to point it at a different release.
4. Cluster continues on, blissfully unaware.
5. CVO gets rescheduled for whatever reason.
6. Cluster pulls a fresh registry image for the new pod, but it's by-tag, so you get the new content.
7. CVO thinks it's still in reconciling mode, because the pullspec hasn't changed.
8. World explodes as the new manifests get applied in a parallel, randomized order.

In the worst case, an attacker who has gained access to the registry can substitute a malicious release image for the tag you were using, and subsequent CVO re-pulls will quietly roll the malicious release payload out to any clusters that are consuming by-tag pullspecs

Scenarios

1. Install a cluster with MCE.
2. Update the cluster to a different version with the MCE web-console.

Acceptance Criteria

1. Post-install and post-update, neither the openshift-cluster-version cluster-version-operator Deployment nor the ClusterVersion resource make any reference to a by-tag pullspec.

Dependencies (internal and external)

Unknown.

Previous Work (Optional):

The move from tags would be a change here:
https://github.com/stolostron/acm-hive-openshift-releases/blob/753913591acdf56ffec0e9f3e3ecb1eba563b3f7/tooling/create-ocp-clusterimagesets.py#L65

You can use the json key "manifest_digest", this will give you a "sha256:....." value that can be used instead of the key.

We also probably want to adjust for the Arch here:
https://github.com/stolostron/acm-hive-openshift-releases/blob/753913591acdf56ffec0e9f3e3ecb1eba563b3f7/tooling/create-ocp-clusterimagesets.py#L46

Open questions:

1. I'm not clear on how ClusterImageSet content are verified for use as installation targets. Is there tooling confirming valid installer or release image signatures? (NO, not in the way your implying, which is why we should make this change)
2. I'm not clear on why ClusterImageSets come into update advice. I'd expect that information to flow through ClusterVersion status's availableUpdates and conditionalUpdates. But if updates are fed by ClusterImageSets today, pivoting to ClusterVersion status sourcing can certainly be punted to follow-up work. (This is available for Hive (IPI) deployed/imported clusters (using Console or Policies), but not on Hosted Control Planes yet)
3. I'm not clear on how existing standalone-cluster updates are performing signature verification today.  I'd expect by-tag pullspecs to fail verification via this code.

Done Checklist

• CI - CI is running, tests are automated and merged.
• Release Enablement <link to Feature Enablement Presentation>
• DEV - Upstream code and tests merged: <link to meaningful PR or GitHub
  Issue>
• DEV - Upstream documentation merged: <link to meaningful PR or GitHub
  Issue>
• DEV - Downstream build attached to advisory: <link to errata>
• QE - Test plans in Polarion: <link or reference to Polarion>
• QE - Automated tests merged: <link or reference to automated tests>
• DOC - Downstream documentation merged: <link to meaningful PR>