When using the --oci-registries-config flag explicitly or getting registries.conf from the environment, execution time when processing related images via the addRelatedImageToMapping function serially can drastically impact performance depending on the number of images involved. In my testing of a large catalog, there were approximately 470 images and this took approximately 13 minutes. This processing occurs prior to letting the underlying oc mirror code plan out the images that should be mirrored. Actual planning time is consistent at around 1 min 30 seconds.
The cause of this is due to the need to determine mirrors for each one of the related images based on the configuration provided in registries.conf, and this action is done serially in a loop. If I introduce parallel execution, the processing time for addRelatedImageToMapping is reduced from ~13 min to ~14 seconds.
Note: the catalog used here is publicly available, but the related images are not so this may be difficult to reproduce.
- Copy catalog image to disk in OCI layout
- Create a ~/.config/containers/registries.conf file with this content
- Create a ISC [path to isc]/isc-registriesconf-performance.yaml
- run oc mirror with OCI flags (running with dry run is sufficient to replicate this issue)
roughly 13 minutes elapses before the planning phase begins
much faster execution before the planning phase begins
I intend to create a PR which adds parallel execution around the addRelatedImageToMapping function