Description of problem:

 
We are encountering an issue with Artifact Endpoint not being passed into recurring runs. Regular runs seem to work fine, but the runs created via scheduleworkflow seem to fail at the "copy-artifacts" step with the error:
Bad value for --endpoint-url "cp": scheme is missing. Must be of the form http://<hostname>/ or https://<hostname>/
This is because the annotations are not being created for the scheduled runs (this is how s3 access info is passed into the tekton tasks). See attached for the different pipelineruns created, first one is a regular run, with annotations filled in for artifact end point etc., the second one was created via recurring run, and does not have any annotations.
 
The issue is an upstream issue due to this:  https://github.com/kubeflow/kfp-tekton/issues/1219
 
 

Prerequisites (if any, like setup, operators/versions):

Steps to Reproduce

1. Deploy a recurring run.
2. Confirm it fails
3. Deploy a regular run confirm it works.
4. Check logs on recurring run pipeline pod with step "copy-artifacts" container and confirm logs match as indicated above.

Actual results:

1. Recurring run fails artifact passing step due to missing endpoint/bucket info:

Bad value for --endpoint-url "cp": scheme is missing. Must be of the form http://<hostname>/ or https://<hostname>/ 

Expected results:

1. Recurring run performs artifact passing successfully.

Reproducibility (Always/Intermittent/Only Once): Always

Build Details:

Workaround:

A possible workaround is to just hardcode the artifact end point / bucket in the artifact script.

Additional info: