-
Bug
-
Resolution: Done-Errata
-
Normal
-
4.13
-
No
-
CFE Sprint 236
-
1
-
False
-
-
Previously, when both `tag` and `digest` were included in container image references, oc-mirror would incorrectly interpret it resulting in an `invalid reference format` error. This issue has been fixed and the images are successfully mirrored.
-
Bug Fix
-
Done
-
Release Notes
Description of problem:
Docker defines image references using the following BNF format (link to source):
Grammar
reference := name [ ":" tag ] [ "@" digest ]
name := [domain '/'] remote-name
domain := host [':' port-number]
host := domain-name | IPv4address | \[ IPv6address \] ; rfc3986 appendix-A
domain-name := domain-component ['.' domain-component]*
domain-component := /([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9])/
port-number := /[0-9]+/
path-component := alpha-numeric [separator alpha-numeric]*
path (or "remote-name") := path-component ['/' path-component]*
alpha-numeric := /[a-z0-9]+/
separator := /[_.]|__|[-]*/
tag := /[\w][\w.-]{0,127}/
digest := digest-algorithm ":" digest-hex
digest-algorithm := digest-algorithm-component [ digest-algorithm-separator digest-algorithm-component ]*
digest-algorithm-separator := /[+.-_]/
digest-algorithm-component := /[A-Za-z][A-Za-z0-9]*/
digest-hex := /[0-9a-fA-F]{32,}/ ; At least 128 bit digest value
identifier := /[a-f0-9]{64}/
Simple examples of valid docker references using the BNF naming convention:
name name:tag name@digest name:tag@digest // <-- this last one is the reason for this issue
stack that lead to this issue:
github.com/openshift/oc/pkg/cli/image/imagesource.ParseReference (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/vendor/github.com/openshift/oc/pkg/cli/image/imagesource/reference.go:111) github.com/openshift/oc-mirror/pkg/image.ParseReference (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/image/image.go:88) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).addRelatedImageToMapping (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/fbc_operators.go:191) github.com/openshift/oc-mirror/pkg/cli/mirror.(*OperatorOptions).plan (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/operator.go:454) github.com/openshift/oc-mirror/pkg/cli/mirror.(*OperatorOptions).run (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/operator.go:128) github.com/openshift/oc-mirror/pkg/cli/mirror.(*OperatorOptions).PlanFull (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/operator.go:63) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).Create.func2 (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/create.go:69) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).run (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/create.go:127) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).Create (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/create.go:73) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).mirrorToMirrorWrapper (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/mirror.go:619) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).mirrorImages (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/mirror.go:371) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).Run (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/mirror.go:312) github.com/openshift/oc-mirror/pkg/cli/mirror.NewMirrorCmd.func1 (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/mirror.go:114) github.com/spf13/cobra.(*Command).execute (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/vendor/github.com/spf13/cobra/command.go:920) github.com/spf13/cobra.(*Command).ExecuteC (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/vendor/github.com/spf13/cobra/command.go:1044) github.com/spf13/cobra.(*Command).Execute (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/vendor/github.com/spf13/cobra/command.go:968) main.main (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/cmd/oc-mirror/main.go:10) runtime.main (/usr/local/Cellar/go@1.19/1.19.8/libexec/src/runtime/proc.go:250) runtime.goexit (/usr/local/Cellar/go@1.19/1.19.8/libexec/src/runtime/asm_amd64.s:1594)
The related image reference in this case starts off as:
cp.icr.io/cp/cpd/postgresql:13.7@sha256:e05434bfdb4b306fbc2e697112e1343907e368eb5f348c1779562d31b9f32ac5
The actual error that is reported can be seen below (note the host is referencing the destination now):
error: "localhost:6000/cp/cpd/postgresql:13.7@sha256" is not a valid image reference: invalid reference format
Note that the digest hex values are not present, which causes the error. However, the root of the problem comes from further up the stack in addRelatedImageToMapping which is calling into the following function:
_, subns, imgName, tag, sha := v1alpha2.ParseImageReference(img.Image)
Since v1alpha2.ParseImageReference is not taking into account the tag plus sha format, it incorrectly arrives at these return values:
registry: "cp.icr.io" org: "cp/cpd" repo: "postgresql" tag: "13.7@sha256" sha: ""
While inside the v1alpha2.ParseImageReference function, the value tmp looks like:
[]string len: 4, cap: 4, ["cp.icr.io","cp","cpd","postgresql:13.7@sha256:e05434bfdb4b306fbc2e697112e1343907e368eb5f348c1779562d31b9f32ac5"]
So when it comes time to process the last entry in tmp (i.e. postgresql:13.7@sha256:e05434bfdb4b306fbc2e697112e1343907e368eb5f348c1779562d31b9f32ac5), the following code executes:
img := strings.Split(tmp[len(tmp)-1], ":")
and you end up with:
[]string len: 3, cap: 3, ["postgresql","13.7@sha256","e05434bfdb4b306fbc2e697112e1343907e368eb5f348c1779562d31b9f32ac5"]
As you can see, the sha256 portion is now separated from its hex value, which is fine, but it means you have to process this situation differently because the @ symbol could be in a different location within the slice. The code currently assumes that the @ symbol will be in index zero of the slice, but in this particular case it's in index one.
In my own PR, I had additional test cases (see https://github.com/openshift/oc-mirror/pull/568/files#diff-4b140d77c1272670b1b508d5c75400d3a2dbf311cdc3489bb0cb7fa5bfe79214) to cover this function. These tests can probably be migrated to a new PR to cover this issue.
Version-Release number of selected component (if applicable):
4.13
How reproducible:
always
Steps to Reproduce:
$ oc mirror -c /Users/jhunkins/.ibm-pak/data/publish/latest/image-set-config.yaml --use-oci-feature --oci-insecure-signature-policy --dest-skip-tls --dest-use-http docker://localhost:6000
Actual results:
Checking push permissions for localhost:6000 Creating directory: oc-mirror-workspace/src/publish Creating directory: oc-mirror-workspace/src/v2 Creating directory: oc-mirror-workspace/src/charts Creating directory: oc-mirror-workspace/src/release-signatures No metadata detected, creating new workspace error: "localhost:6000/cp/cpd/postgresql:13.7@sha256" is not a valid image reference: invalid reference format
Expected results:
no parsing error
Additional info:
- blocks
-
-
- Closed
-
- is cloned by
-
-
- Closed
-
- links to
-
RHSA-2023:5006
OpenShift Container Platform 4.14.z security update
