Details
Description
This is a clone of issue OCPBUGS-11840. The following is the description of the original issue:
—
Description of problem:
Docker defines image references using the following BNF format (link to source):
Grammar
reference := name [ ":" tag ] [ "@" digest ]
name := [domain '/'] remote-name
domain := host [':' port-number]
host := domain-name | IPv4address | \[ IPv6address \] ; rfc3986 appendix-A
domain-name := domain-component ['.' domain-component]*
domain-component := /([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9])/
port-number := /[0-9]+/
path-component := alpha-numeric [separator alpha-numeric]*
path (or "remote-name") := path-component ['/' path-component]*
alpha-numeric := /[a-z0-9]+/
separator := /[_.]|__|[-]*/
tag := /[\w][\w.-]{0,127}/
digest := digest-algorithm ":" digest-hex
digest-algorithm := digest-algorithm-component [ digest-algorithm-separator digest-algorithm-component ]*
digest-algorithm-separator := /[+.-_]/
digest-algorithm-component := /[A-Za-z][A-Za-z0-9]*/
digest-hex := /[0-9a-fA-F]{32,}/ ; At least 128 bit digest value
identifier := /[a-f0-9]{64}/
Simple examples of valid docker references using the BNF naming convention:
name name:tag name@digest name:tag@digest // <-- this last one is the reason for this issue
stack that lead to this issue:
github.com/openshift/oc/pkg/cli/image/imagesource.ParseReference (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/vendor/github.com/openshift/oc/pkg/cli/image/imagesource/reference.go:111) github.com/openshift/oc-mirror/pkg/image.ParseReference (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/image/image.go:88) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).addRelatedImageToMapping (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/fbc_operators.go:191) github.com/openshift/oc-mirror/pkg/cli/mirror.(*OperatorOptions).plan (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/operator.go:454) github.com/openshift/oc-mirror/pkg/cli/mirror.(*OperatorOptions).run (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/operator.go:128) github.com/openshift/oc-mirror/pkg/cli/mirror.(*OperatorOptions).PlanFull (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/operator.go:63) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).Create.func2 (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/create.go:69) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).run (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/create.go:127) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).Create (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/create.go:73) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).mirrorToMirrorWrapper (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/mirror.go:619) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).mirrorImages (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/mirror.go:371) github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).Run (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/mirror.go:312) github.com/openshift/oc-mirror/pkg/cli/mirror.NewMirrorCmd.func1 (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/pkg/cli/mirror/mirror.go:114) github.com/spf13/cobra.(*Command).execute (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/vendor/github.com/spf13/cobra/command.go:920) github.com/spf13/cobra.(*Command).ExecuteC (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/vendor/github.com/spf13/cobra/command.go:1044) github.com/spf13/cobra.(*Command).Execute (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/vendor/github.com/spf13/cobra/command.go:968) main.main (/Users/jhunkins/go/src/github.com/jchunkins/oc-mirror/cmd/oc-mirror/main.go:10) runtime.main (/usr/local/Cellar/go@1.19/1.19.8/libexec/src/runtime/proc.go:250) runtime.goexit (/usr/local/Cellar/go@1.19/1.19.8/libexec/src/runtime/asm_amd64.s:1594)
The related image reference in this case starts off as:
cp.icr.io/cp/cpd/postgresql:13.7@sha256:e05434bfdb4b306fbc2e697112e1343907e368eb5f348c1779562d31b9f32ac5
The actual error that is reported can be seen below (note the host is referencing the destination now):
error: "localhost:6000/cp/cpd/postgresql:13.7@sha256" is not a valid image reference: invalid reference format
Note that the digest hex values are not present, which causes the error. However, the root of the problem comes from further up the stack in addRelatedImageToMapping which is calling into the following function:
_, subns, imgName, tag, sha := v1alpha2.ParseImageReference(img.Image)
Since v1alpha2.ParseImageReference is not taking into account the tag plus sha format, it incorrectly arrives at these return values:
registry: "cp.icr.io" org: "cp/cpd" repo: "postgresql" tag: "13.7@sha256" sha: ""
While inside the v1alpha2.ParseImageReference function, the value tmp looks like:
[]string len: 4, cap: 4, ["cp.icr.io","cp","cpd","postgresql:13.7@sha256:e05434bfdb4b306fbc2e697112e1343907e368eb5f348c1779562d31b9f32ac5"]
So when it comes time to process the last entry in tmp (i.e. postgresql:13.7@sha256:e05434bfdb4b306fbc2e697112e1343907e368eb5f348c1779562d31b9f32ac5), the following code executes:
img := strings.Split(tmp[len(tmp)-1], ":")
and you end up with:
[]string len: 3, cap: 3, ["postgresql","13.7@sha256","e05434bfdb4b306fbc2e697112e1343907e368eb5f348c1779562d31b9f32ac5"]
As you can see, the sha256 portion is now separated from its hex value, which is fine, but it means you have to process this situation differently because the @ symbol could be in a different location within the slice. The code currently assumes that the @ symbol will be in index zero of the slice, but in this particular case it's in index one.
In my own PR, I had additional test cases (see https://github.com/openshift/oc-mirror/pull/568/files#diff-4b140d77c1272670b1b508d5c75400d3a2dbf311cdc3489bb0cb7fa5bfe79214) to cover this function. These tests can probably be migrated to a new PR to cover this issue.
Version-Release number of selected component (if applicable):
4.13
How reproducible:
always
Steps to Reproduce:
$ oc mirror -c /Users/jhunkins/.ibm-pak/data/publish/latest/image-set-config.yaml --use-oci-feature --oci-insecure-signature-policy --dest-skip-tls --dest-use-http docker://localhost:6000
Actual results:
Checking push permissions for localhost:6000 Creating directory: oc-mirror-workspace/src/publish Creating directory: oc-mirror-workspace/src/v2 Creating directory: oc-mirror-workspace/src/charts Creating directory: oc-mirror-workspace/src/release-signatures No metadata detected, creating new workspace error: "localhost:6000/cp/cpd/postgresql:13.7@sha256" is not a valid image reference: invalid reference format
Expected results:
no parsing error
Additional info:
Attachments
Issue Links
- clones
-
OCPBUGS-11840 Invalid docker ref parsing when tag and sha are both provided
-
- Closed
-
- is blocked by
-
-
- Closed
-
- is cloned by
-
-
- Closed
-
- links to
