Customer has a disconnected OCP 4.16 environment(issue can be reproduced locally as well). He configures imagetagset configuration as shown below. Problem only happens when optional_namespace is used as mentioned in the documentation

Illustration: <registry>/<optional-namespace>/oss/kubernetes/pause

It works if he mirrors without  the "optional-namespace" ie  <registry>/oss/kubernetes/pause
 
apiVersion: config.openshift.io/v1
kind: ImageTagMirrorSet
metadata:
  name: mirrorset
spec:
  imageTagMirrors:
  - mirrorSourcePolicy: NeverContactSource
    mirrors:
    - docker.<registry>/<optional_namespace>/oss/kubernetes/pause
    source: mcr.microsoft.com/oss/kubernetes/pause

    
This ends up creating proper hosts.toml file
 
 
server = "<registry>"
[host."<registry>"]
  capabilities = ["pull", "resolve"]

 
Containerd fails to pull the pause image with the following error "not found"
 
Failed to create pod sandbox: rpc error: code = NotFound desc = failed to get sandbox image "mcr.microsoft.com/oss/kubernetes/pause:3.9": failed to pull image "mcr.microsoft.com/oss/kubernetes/pause:3.9": failed to pull and unpack image "mcr.microsoft.com/oss/kubernetes/pause:3.9": failed to resolve reference "mcr.microsoft.com/oss/kubernetes/pause:3.9": mcr.microsoft.com/oss/kubernetes/pause:3.9: not found
 
 
Containerd in the background is executing a similar command to check for the repositories
 
curl -v <registry>/<optional_namespace>/v2/oss/kubernetes/pause/manifests/3.9?ns=mcr.microsoft.com
 
 
with optional_namespaces, a simple docker registry or quay registry is not returning the repo outputs. Due to this, we are getting "not found" when optional namespaces are used in the mirror registry
Customer can pull the image directly 
 
C:\scripts> .\crictl.exe -debug pull <registry>/<optional_namespace>/oss/kubernetes/pause:3.9
Image is up to date for sha256:c868a912caad9a64e6601bafa5188aa9d5d981b0c63f58988d90c88160c19688
 
  

    OCP 4.16.24/WMCO 10.16.1

    Always

    1. Create a ImageTagMirrorSet definition for pause container image for use in windows    
    2. try to pull the image using mcr.microsoft.com/oss/kubernetes/pause:3.9
    3.
    

With optional_namespaces, containerd fails to pull the image    

   As explained in the documentation, containerd should be able to pull the image with optional namespaces in use