Description

Epic Overview

Zstd compression is a more efficient and faster process than gzip. Container tools have the ability to pull Zstd compressed images. But pushing or converting a manifest list with gzip images into Zstd involves lot of manual effort and inconvenience.

We want to enable a feature which allows end-users to create a manifest list with images compressed in both `gzip` and `zstd`easily.

RHEL 8.9/9.3 Scope of work:

Before RHEL 8.9/9.3, changes were made to the Podman/Buildah/Skopeo commands that push and pull images.  These changes allowed these utilities to push or pull, as appropriate, Zstd compressed images created by other container tools such as Docker.  This ability was not documented, and it should be noted that it is available in these releases of RHEL.

Note before RHEL 8.9/9.3, the Podman/Buildah/Skopeo tools did not have the ability to create an image using Zstd compression.  These tools could only push/pull those images compressed with Zstd.  For RHEL 8.9/9.3, the Podman and Buildah tools can now create images and compress them with Zstd compression.  This ability also needs to be release noted, and documented as appropriate.

RHEL 8.10/9.4 Scope of work:

None, pushing to RHEL 9.5/10.0 beta

RHEL 9.5/10.0 beta Scope of work:

In addition to Zstd compression, there is a variant called `zstd:chunked.  This variant allows partial images to be pulled/pushed as appropriate.  This functionality will not be part of the RHEL 8.9/9.3 work but is currently planned for RHEL 8.10 and 9.4.

Goals

By itself zstd compression is faster and more efficient than gzip. It also supports partial pulls which in some cases can dramatically reduce the amount of network traffic and storage involved in pulling images.

This pulls together the final pieces needed for customers to take advantage of that savings in a convenient way.

Requirements

A list of specific needs or objectives that a Feature must deliver to satisfy the Feature.. Some requirements will be flagged as MVP. If an MVP gets shifted, the feature shifts.  If a non MVP requirement slips, it does not shift the feature.

(Optional) Use Cases

< How will the user interact with this feature? >

< Which users will use this and when will they use it? >

< Is this feature used as part of current user interface? >

Out of Scope

Background, and strategic fit

< What does the person writing code, testing, documenting need to know? >

Assumptions

< Are there assumptions being made regarding prerequisites and dependencies?>

< Are there assumptions about hardware, software or people resources?>

Customer Considerations

< Are there specific customer environments that need to be considered (such as working with existing h/w and software)?>

<Are there Upgrade considerations that customers need to account for, or that the Feature should address on behalf of the customer?>

<Does the Feature introduce data that could be gathered and used for Insights purposes?>

Documentation Considerations

< What educational or reference material (docs) is required to support this product feature? For users/admins? Other functions (security officers, etc)? >

<What does success look like?>

< Does this feature have doc impact?  Possible values are: New Content, Updates to existing content,  Release Note, or No Doc Impact>

 <If unsure and no Technical Writer is available, please contact Content Strategy. If yes, complete the following.>

• <What concepts do customers need to understand to be successful in [action]?>
• <How do we expect customers will use the feature? For what purpose(s)?>
• <What reference material might a customer want/need to complete [action]?>
• <Is there source material that can be used as reference for the Technical Writer in writing the content? If yes, please link if available. >
• <What is the doc impact (New Content, Updates to existing content, or Release Note)?>

Interoperability Considerations

< Which other products and versions in our portfolio does this feature impact? >

<If other products will be impacted set the ‘LP_Interop’ label on the Feature>

< What interoperability test scenarios should be factored by the layered product(s)? >

Questions

This is available in the `podman push`, `podman manifest`, and `podman manifest push` commands and should be tested for all three.  The testing used for `zstd` can be extrapolated for `zstd:chunked`, at least in theory.

Update April 15, 2024

From gscrivan@redhat.com pertaining to the final piece of work, Zstd:chunked

 
For an end-user point of view, not much should change, once they build
images in the new format with `--compression-format zstd:chunked` and
push them to a registry, then Podman should pull (assuming the registry
supports HTTP Range requests) only the missing files.

e.g.

push some images in the new format:

$ podman push --compression-format zstd:chunked quay.io/giuseppe/zstd-chunked:fedora
$ podman push --compression-format zstd:chunked quay.io/giuseppe/zstd-chunked:nginx

then pull them from a different storage/system:

1. podman pull quay.io/giuseppe/zstd-chunked:fedora
   Trying to pull quay.io/giuseppe/zstd-chunked:fedora...
   Getting image source signatures
   Copying blob 39be57add615 done 69.2MiB / 69.2MiB (skipped: 62.9KiB = 0.09%)
   Copying config 0567d09352 done |
   Writing manifest to image destination
   6ef1e5824ef9e3417d7759a6a2cf2d1e20d36a0fb1aa3384ed7fc0ecfb77ec97

1. bin/podman pull quay.io/giuseppe/zstd-chunked:nginx
   Trying to pull quay.io/giuseppe/zstd-chunked:nginx...
   Getting image source signatures
   Copying blob 18108aa4ae15 done 82.2MiB / 82.2MiB (skipped: 66.7MiB = 81.19%)
   Copying config f09210ecf0 done |
   Writing manifest to image destination
   4131cda15e1b8336c446963ae78144be643f170e143ed836b27f247aed57e7c3

From the QE point of view, the testing should be to create some images
in the new format, and make sure the layers share some files; then pull
them on a different system or with a fresh storage and see that
deduplication is happening from the "skipped: .*" output from pull.
Also they need to make sure that if the feature is disabled, then the
old mechanism is in place.

The partial pull mechanism is customizable from the storage.conf file:

[storage.options]

pull_options = {convert_images = "false", enable_partial_images = "true", use_hard_links = "false", ostree_repos=""}

These values are already documented in the containers-storage.conf.5 man
page:

containers/storage supports four keys

• enable_partial_images="true" | "false"
  Tells containers/storage to look for files previously pulled in storage
  rather then always pulling them from the container registry.
• use_hard_links = "false" | "true"
  Tells containers/storage to use hard links rather then create new files in
  the image, if an identical file already existed in storage.
• ostree_repos = ""
  Tells containers/storage where an ostree repository exists that might have
  previously pulled content which can be used when attempting to avoid
  pulling content from the container registry
• convert_images = "false" | "true"
  If set to true, containers/storage will convert images to a format compatible with
  partial pulls in order to take advantage of local deduplication and hardlinking. It is an
  expensive operation so it is not enabled by default.

SME: Miloslav Trmac and Giuseppe Scrivano