Description of problem:

There is a race condition in the Tekton Results Watcher, where it tries to create an object which already exists. This should not happen as there is unique constraint on that column. To the best of my understanding there is no data loss, but the object is re-queued and few error messages are logged which is confusing. The issue happens with single controller (multiple workers) and even more so with more replicas, which further complicates the horizontal scale of the Watcher.

The error in the logs is:

{"level":"debug","time":"2025-09-30T15:23:09.495Z","logger":"watcher","caller":"dynamic/dynamic.go:152","msg":"Error upserting record to API server","commit":"385423f-dirty","knative.dev/pod":"tekton-results-watcher-854b5cb4b4-9jq2h","knative.dev/controller":"github.com.tektoncd.results.pkg.watcher.reconciler.pipelinerun.Reconciler","knative.dev/kind":"tekton.dev.PipelineRun","knative.dev/traceid":"0f07c6cd-4d93-4353-a75c-f5be97d10972","knative.dev/key":"default/test-pipelinerun-foo-8hhr5","results.tekton.dev/kind":"PipelineRun","error":"rpc error: code = Unknown desc = ERROR: duplicate key value violates unique constraint \"results_by_name\" (SQLSTATE 23505)","results.tekton.dev/time-taken-ms":4} {"level":"debug","time":"2025-09-30T15:23:09.495Z","logger":"watcher","caller":"dynamic/dynamic.go:124","msg":"Leaving dynamic Reconciler normally with context properly canceled","commit":"385423f-dirty","knative.dev/pod":"tekton-results-watcher-854b5cb4b4-9jq2h","knative.dev/controller":"github.com.tektoncd.results.pkg.watcher.reconciler.pipelinerun.Reconciler","knative.dev/kind":"tekton.dev.PipelineRun","knative.dev/traceid":"0f07c6cd-4d93-4353-a75c-f5be97d10972","knative.dev/key":"default/test-pipelinerun-foo-8hhr5","results.tekton.dev/kind":"PipelineRun"} {"level":"error","time":"2025-09-30T15:23:09.495Z","logger":"watcher","caller":"pipelinerun/reconciler.go:313","msg":"Returned an error","commit":"385423f-dirty","knative.dev/pod":"tekton-results-watcher-854b5cb4b4-9jq2h","knative.dev/controller":"github.com.tektoncd.results.pkg.watcher.reconciler.pipelinerun.Reconciler","knative.dev/kind":"tekton.dev.PipelineRun","knative.dev/traceid":"0f07c6cd-4d93-4353-a75c-f5be97d10972","knative.dev/key":"default/test-pipelinerun-foo-8hhr5","targetMethod":"ReconcileKind","error":"error upserting record: rpc error: code = Unknown desc = ERROR: duplicate key value violates unique constraint \"results_by_name\" (SQLSTATE 23505)","stacktrace":"github.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1/pipelinerun.(*reconcilerImpl).Reconcile\n\tgithub.com/tektoncd/pipeline@v1.3.1/pkg/client/injection/reconciler/pipeline/v1/pipelinerun/reconciler.go:313\nknative.dev/pkg/controller.(*Impl).processNextWorkItem\n\tknative.dev/pkg@v0.0.0-20250415155312-ed3e2158b883/controller/controller.go:521\nknative.dev/pkg/controller.(*Impl).RunContext.func3\n\tknative.dev/pkg@v0.0.0-20250415155312-ed3e2158b883/controller/controller.go:478"} {"level":"error","time":"2025-09-30T15:23:09.495Z","logger":"watcher","caller":"controller/controller.go:545","msg":"Reconcile error","commit":"385423f-dirty","knative.dev/pod":"tekton-results-watcher-854b5cb4b4-9jq2h","knative.dev/controller":"github.com.tektoncd.results.pkg.watcher.reconciler.pipelinerun.Reconciler","knative.dev/kind":"tekton.dev.PipelineRun","knative.dev/traceid":"0f07c6cd-4d93-4353-a75c-f5be97d10972","knative.dev/key":"default/test-pipelinerun-foo-8hhr5","duration":"4.69486ms","error":"error upserting record: rpc error: code = Unknown desc = ERROR: duplicate key value violates unique constraint \"results_by_name\" (SQLSTATE 23505)","stacktrace":"knative.dev/pkg/controller.(*Impl).handleErr\n\tknative.dev/pkg@v0.0.0-20250415155312-ed3e2158b883/controller/controller.go:545\nknative.dev/pkg/controller.(*Impl).processNextWorkItem\n\tknative.dev/pkg@v0.0.0-20250415155312-ed3e2158b883/controller/controller.go:522\nknative.dev/pkg/controller.(*Impl).RunContext.func3\n\tknative.dev/pkg@v0.0.0-20250415155312-ed3e2158b883/controller/controller.go:478"} {"level":"debug","time":"2025-09-30T15:23:09.495Z","logger":"watcher","caller":"controller/controller.go:553","msg":"Requeuing key default/test-pipelinerun-foo-8hhr5 due to non-permanent error (depth: 0)","commit":"385423f-dirty","knative.dev/pod":"tekton-results-watcher-854b5cb4b4-9jq2h","knative.dev/controller":"github.com.tektoncd.results.pkg.watcher.reconciler.pipelinerun.Reconciler","knative.dev/kind":"tekton.dev.PipelineRun","knative.dev/traceid":"0f07c6cd-4d93-4353-a75c-f5be97d10972","knative.dev/key":"default/test-pipelinerun-foo-8hhr5"}

Workaround

There is no workaround for the issue. 

Prerequisites (if any, like setup, operators/versions):

Steps to Reproduce

 # <steps>

 

Actual results:

Expected results:

Reproducibility (Always/Intermittent/Only Once):

Acceptance criteria: 

 

Definition of Done:

Build Details:

Additional info (Such as Logs, Screenshots, etc):

 

 *