-
Bug
-
Resolution: Unresolved
-
Normal
-
Pipelines 1.13.1, Pipelines 1.13.0, Pipelines 1.14.0, Pipelines 1.14.1
-
False
-
None
-
False
-
-
Description of problem:
When a user attempts to create a Pipeline that contains a Task which contains a script with a bad variable reference that Tekton cannot parse, the validation code panics, causing the webhook call to fail without a good explanation. The user is only presented with:
... one or more objects failed to apply, reason: Internal error occurred: failed calling webhook "validation.webhook.pipeline.tekton.dev": failed to call webhook: Post "https://tekton-pipelines-webhook.openshift-pipelines.svc:443/resource-validation?timeout=10s": EOF
Upstream issue link: Tekton pipeline validation crashes on script validation in Openshift · Issue #7756 · tektoncd/pipeline (github.com)
Prerequisites (if any, like setup, operators/versions):
OpenShift Pipelines v1.13.x, v1.14.0, v1.14.1
*Client version: 0.28.0 Chains version: v0.20.0 Pipeline version: v0.56.1 Triggers version: v0.26.1 Operator version: v0.70.1*
Steps to Reproduce
- Create a Task with a script that contains a bad variable reference that Tekton does not understand like $(new_image)
- The user meant to write ${new_image} to refer to a shell variable in the script, but used parentheses by accident
- Write a Pipeline that calls this Task
- Try to add the Pipeline to your cluster with oc create|apply -f pipeline.yaml
- Observe the error mentioned above
Actual results:
The validation code panics in the background, and tekton-pipeline-webhook Pod logs show this ...
{"severity":"error","timestamp":"2024-03-14T14:30:22.136Z","logger":"tekton-pipelines-webhook","caller":"webhook/webhook.go:237","message":"http: panic serving 10.128.0.1:43866: runtime error: index out of range [1] with length 1\ngoroutine 1045 [running]:\nnet/http.(*conn).serve.func1()\n\t/usr/lib/golang/src/net/http/server.go:1854 +0xbf\npanic({0x1f79980, 0xc007fe8a80})\n\t/usr/lib/golang/src/runtime/panic.go:890 +0x263\ngithub.com/tektoncd/pipeline/pkg/apis/pipeline/v1.validateMatrixedPipelineTaskConsumed({0xc007fe1860?, 0x3, 0x20c8ffe?}, 0x6?)\n\t/go/src/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1/pipeline_validation.go:791 +0x3b8\ngithub.com/tektoncd/pipeline/pkg/apis/pipeline/v1.findAndValidateResultRefsForMatrix({0xc008068000?, 0xd, 0x0?}, 0x30222dd200000000?)\n\t/go/src/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1/pipeline_validation.go:773 +0x292\ngithub.com/tektoncd/pipeline/pkg/apis/pipeline/v1.validateTaskResultsFromMatrixedPipelineTasksConsumed({0xc008068000, 0xd, 0xd})\n\t/go/src/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1/pipeline_validation.go:809 +0x189\ngithub.com/tektoncd/pipeline/pkg/apis/pipeline/v1.validateMatrix({0x23c75d0, 0xc007fe0cf0}, {0xc008068000?, 0xd, 0xd})\n\t/go/src/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1/pipeline_validation.go:761 +0x137\ngithub.com/tektoncd/pipeline/pkg/apis/pipeline/v1.(*PipelineSpec).Validate(0xc005b68fa8, {0x23c75d0, 0xc007fe0cf0})\n\t/go/src/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1/pipeline_validation.go:89 +0x69b\ngithub.com/tektoncd/pipeline/pkg/apis/pipeline/v1.(*Pipeline).Validate(0xc005b68ea0, {0x23c75d0, 0xc007fe0cc0})\n\t/go/src/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1/pipeline_validation.go:56 +0xba\nknative.dev/pkg/webhook/resourcesemantics/validation.validate({0x23c75d0, 0xc007fe0cc0}, {0x23c7cd0, 0xc005b68ea0}, 0xc005b68d00)\n\t/go/src/github.com/tektoncd/pipeline/vendor/knative.dev/pkg/webhook/resourcesemantics/validation/validation_admit.go:182 +0x127\nknative.dev/pkg/webhook/resourcesemantics/validation.(*reconciler).Admit(0xc000001440, {0x23c75d0?, 0xc007fe0690?}, 0xc005b68d00)\n\t/go/src/github.com/tektoncd/pipeline/vendor/knative.dev/pkg/webhook/resourcesemantics/validation/validation_admit.go:79 +0x1c5\nknative.dev/pkg/webhook.admissionHandler.func1({0x23c63e0?, 0xc007d635e0}, 0xc007e1db00)\n\t/go/src/github.com/tektoncd/pipeline/vendor/knative.dev/pkg/webhook/admission.go:123 +0x7fb\nnet/http.HandlerFunc.ServeHTTP(0xc00565fa38?, {0x23c63e0?, 0xc007d635e0?}, 0x6b1005?)\n\t/usr/lib/golang/src/net/http/server.go:2122 +0x2f\nnet/http.(*ServeMux).ServeHTTP(0xc007fe04b0?, {0x23c63e0, 0xc007d635e0}, 0xc007e1db00)\n\t/usr/lib/golang/src/net/http/server.go:2500 +0x149\nknative.dev/pkg/webhook.(*Webhook).ServeHTTP(0xc000000c00, {0x23c63e0, 0xc007d635e0}, 0xc007e1db00)\n\t/go/src/github.com/tektoncd/pipeline/vendor/knative.dev/pkg/webhook/webhook.go:320 +0xb3\nknative.dev/pkg/network/handlers.(*Drainer).ServeHTTP(0xc0001c4690, {0x23c63e0, 0xc007d635e0}, 0xc007e1db00)\n\t/go/src/github.com/tektoncd/pipeline/vendor/knative.dev/pkg/network/handlers/drain.go:113 +0x15e\nnet/http.serverHandler.ServeHTTP({0x23b7340?}, {0x23c63e0, 0xc007d635e0}, 0xc007e1db00)\n\t/usr/lib/golang/src/net/http/server.go:2936 +0x316\nnet/http.(*conn).serve(0xc0055b47e0, {0x23c75d0, 0xc00065ef00})\n\t/usr/lib/golang/src/net/http/server.go:1995 +0x612\ncreated by net/http.(*Server).Serve\n\t/usr/lib/golang/src/net/http/server.go:3089 +0x5ed\n","commit":"9be03e2","stacktrace":"knative.dev/pkg/webhook.(*zapWrapper).Write\n\t/go/src/github.com/tektoncd/pipeline/vendor/knative.dev/pkg/webhook/webhook.go:237\nlog.(*Logger).Output\n\t/usr/lib/golang/src/log/log.go:194\nlog.(*Logger).Printf\n\t/usr/lib/golang/src/log/log.go:204\nnet/http.(*Server).logf\n\t/usr/lib/golang/src/net/http/server.go:3215\nnet/http.(*conn).serve.func1\n\t/usr/lib/golang/src/net/http/server.go:1855\nruntime.gopanic\n\t/usr/lib/golang/src/runtime/panic.go:890\nruntime.goPanicIndex\n\t/usr/lib/golang/src/runtime/panic.go:113\ngithub.com/tektoncd/pipeline/pkg/apis/pipeline/v1.validateMatrixedPipelineTaskConsumed\n\t/go/src/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1/pipeline_validation.go:791\ngithub.com/tektoncd/pipeline/pkg/apis/pipeline/v1.findAndValidateResultRefsForMatrix\n\t/go/src/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1/pipeline_validation.go:773\ngithub.com/tektoncd/pipeline/pkg/apis/pipeline/v1.validateTaskResultsFromMatrixedPipelineTasksConsumed\n\t/go/src/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1/pipeline_validation.go:809\ngithub.com/tektoncd/pipeline/pkg/apis/pipeline/v1.validateMatrix\n\t/go/src/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1/pipeline_validation.go:761\ngithub.com/tektoncd/pipeline/pkg/apis/pipeline/v1.(*PipelineSpec).Validate\n\t/go/src/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1/pipeline_validation.go:89\ngithub.com/tektoncd/pipeline/pkg/apis/pipeline/v1.(*Pipeline).Validate\n\t/go/src/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1/pipeline_validation.go:56\nknative.dev/pkg/webhook/resourcesemantics/validation.validate\n\t/go/src/github.com/tektoncd/pipeline/vendor/knative.dev/pkg/webhook/resourcesemantics/validation/validation_admit.go:182\nknative.dev/pkg/webhook/resourcesemantics/validation.(*reconciler).Admit\n\t/go/src/github.com/tektoncd/pipeline/vendor/knative.dev/pkg/webhook/resourcesemantics/validation/validation_admit.go:79\nknative.dev/pkg/webhook.admissionHandler.func1\n\t/go/src/github.com/tektoncd/pipeline/vendor/knative.dev/pkg/webhook/admission.go:123\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2122\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2500\nknative.dev/pkg/webhook.(*Webhook).ServeHTTP\n\t/go/src/github.com/tektoncd/pipeline/vendor/knative.dev/pkg/webhook/webhook.go:320\nknative.dev/pkg/network/handlers.(*Drainer).ServeHTTP\n\t/go/src/github.com/tektoncd/pipeline/vendor/knative.dev/pkg/network/handlers/drain.go:113\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/golang/src/net/http/server.go:2936\nnet/http.(*conn).serve\n\t/usr/lib/golang/src/net/http/server.go:1995"}
and the user is presented with this error message:
...one or more objects failed to apply, reason: Internal error occurred: failed calling webhook "validation.webhook.pipeline.tekton.dev": failed to call webhook: Post "https://tekton-pipelines-webhook.openshift-pipelines.svc:443/resource-validation?timeout=10s": EOF
Expected results:
The validation code should not panic, and a message about what is wrong in the Pipeline YAML should be presented to the user. In this case, perhaps it should say something about an invalid variable reference and print the bad reference '$(new_image)'. It could go further to tell the user what valid variable references must start with, like $(params...) or $(workspaces...) etc.
Reproducibility (Always/Intermittent/Only Once):
It seems to be triggered by any Pipeline that contains a Task which contains a script with an invalid variable reference like $(...), where '...' is not something that can be resolved properly by Tekton. Valid variable references are listed here: Tekton Variables
Acceptance criteria:
Definition of Done:
Build Details:
Additional info (Such as Logs, Screenshots, etc):