Uploaded image for project: 'OpenShift Pipelines'
  1. OpenShift Pipelines
  2. SRVKP-10094

Triggering a new PipelineRun causes previously expired results (past log retention) to be populated again.

XMLWordPrintable

    • 1
    • False
    • Hide

      None

      Show
      None
    • False
    • Release Note Not Required

      Description of problem:

      Triggering a new PipelineRun causes previously expired results (past results retention) to be populated again.

      Prerequisites (if any, like setup, operators/versions):

      **

      1. Install 1.21 OSP
      2. Disable old pruner & enable tektonpruner with config as below
      tektonpruner:
          disabled: false
          global-config:
            enforcedConfigLevel: namespace
            failedHistoryLimit: 7
            historyLimit: 10
            namespaces:
              test1:
                failedHistoryLimit: 3
                successfulHistoryLimit: 2
            successfulHistoryLimit: 3
          options: {}
       
      1. Also enable results retention with below config
      result:
          auth_disable: true
          disabled: false
          is_external_db: false
          log_level: debug
          loki_stack_name: logging-loki
          loki_stack_namespace: openshift-logging
          options:
            configMaps:
              tekton-results-config-results-retention-policy:
                data:
                  defaultRetention: 30m
                  policies: |
                    - name: "short-term-test-retention"
                      selector:
                        matchNamespaces:
                          - "test1"
                      retention: "3m"
                  runAt: '*/1 * * * *'
                metadata:
                  creationTimestamp: null 

      Steps to Reproduce

      1. Create few pipelines and trigger pipelineRuns and validate results are populated with opc results command
      $ oc create -f ./testdata/ecosystem/pipelines/
      pipeline.tekton.dev/buildah-ns-pipeline created
      pipeline.tekton.dev/buildah-pipeline created
      pipeline.tekton.dev/caches-python-pipeline created
      pipeline.tekton.dev/git-cli-read-private-pipeline created
      pipeline.tekton.dev/git-cli-pipeline created
      pipeline.tekton.dev/git-clone-read-private-pipeline created
      pipeline.tekton.dev/helm-upgrade-from-repo created
      pipeline.tekton.dev/helm-upgrade-from-source created
      pipeline.tekton.dev/jib-maven-pipeline created
      pipeline.tekton.dev/maven-pipeline created
      pipeline.tekton.dev/nodejs-ex-git created
      pipeline.tekton.dev/opc-task-pipeline created
      pipeline.tekton.dev/pull-request-pipeline created
      pipeline.tekton.dev/s2i-dotnet-pipeline created
      pipeline.tekton.dev/s2i-go-pipeline created
      pipeline.tekton.dev/s2i-java-pipeline created
      pipeline.tekton.dev/s2i-nodejs-pipeline created
      pipeline.tekton.dev/s2i-perl-pipeline created
      pipeline.tekton.dev/s2i-php-pipeline created
      pipeline.tekton.dev/s2i-python-pipeline created
      pipeline.tekton.dev/s2i-ruby-pipeline created
      
      [10:21:56] smanthin@smanthin-thinkpadp16vgen1:~/Automation/release-tests$ oc create -f ./testdata/pvc/pvc.yaml
      persistentvolumeclaim/shared-pvc created
      
      [10:22:03] smanthin@smanthin-thinkpadp16vgen1:~/Automation/release-tests$ oc create -f ./testdata/ecosystem/pipelineruns/
      pipelinerun.tekton.dev/buildah-disconnected-run created
      pipelinerun.tekton.dev/buildah-ns-run created
      pipelinerun.tekton.dev/buildah-run created
      pipelinerun.tekton.dev/git-cli-read-private-sa-run created
      pipelinerun.tekton.dev/git-cli-read-private-run created
      pipelinerun.tekton.dev/git-cli-run created
      pipelinerun.tekton.dev/git-clone-read-private-pipeline-sa-run created
      pipelinerun.tekton.dev/git-clone-read-private-pipeline-run created
      pipelinerun.tekton.dev/git-clone-stepaction-run created
      pipelinerun.tekton.dev/helm-upgrade-from-repo-run created
      pipelinerun.tekton.dev/helm-upgrade-from-source-run created
      pipelinerun.tekton.dev/jib-maven-pz-run created
      pipelinerun.tekton.dev/jib-maven-run created
      pipelinerun.tekton.dev/kn-apply-pz-run created
      pipelinerun.tekton.dev/kn-apply-run created
      pipelinerun.tekton.dev/kn-pz-run created
      pipelinerun.tekton.dev/kn-run created
      pipelinerun.tekton.dev/maven-run4 created
      pipelinerun.tekton.dev/nodejs-ex-git-pr created
      pipelinerun.tekton.dev/opc-task-run created
      pipelinerun.tekton.dev/openshift-client-run created
      pipelinerun.tekton.dev/pull-request-pipeline-run created
      pipelinerun.tekton.dev/skopeo-copy-run created
      pipelinerun.tekton.dev/tkn-pac-run-2 created
      pipelinerun.tekton.dev/tkn-version-run created
      pipelinerun.tekton.dev/tkn-run-2 created 
      1. Get the list of pipelineRuns along with results list
      $ oc get pipelineruns -n test1 -o json | jq -r '
        .items[]
        | [
            .metadata.name,
            (.metadata.annotations["results.tekton.dev/result"] // "N/A")
          ]
        | @tsv
      '
      buildah-ns-run    test1/results/70482af9-9d8e-46c5-9a0d-03b5042eb500
      buildah-run    test1/results/caaa9684-5869-4100-912d-6d8ec83e925a
      jib-maven-pz-run    test1/results/d2fd3432-4b10-4a13-b85e-88dffa0c2d29
      jib-maven-run    test1/results/81c08293-fca6-4447-9fb2-49f7fbf28988
      kn-pz-run    test1/results/a3c19b66-afb4-476a-be05-8917f9a0b257
      kn-run    test1/results/a228c871-d6d1-4fb5-a14e-e0319cea60e5
      maven-run4    test1/results/563db362-8fd7-4b96-87b4-c5a100a593a4
      nodejs-ex-git-pr    test1/results/36465a4c-0a48-4c32-ba3c-16af06fd9e83
      pull-request-pipeline-run    test1/results/99516f5e-b206-4bc5-a6c3-87dc357120f4
      tkn-pac-run-2    test1/results/bc8a11bc-85cf-4647-a4ec-e3334e5b9d03
      tkn-run-2    test1/results/df751fda-8892-4279-b438-935d4567daac 
      
      
      
      $ opc results result list --insecure --addr ${RESULTS_API} test1
      Name                                                Start                                   Update
      test1/results/563db362-8fd7-4b96-87b4-c5a100a593a4  2026-01-08 12:26:28 +0530 IST           2026-01-08 12:27:22 +0530 IST
      test1/results/156a2910-44fc-4b1e-bad8-b21fc29c911b  2026-01-08 12:26:23 +0530 IST           2026-01-08 12:26:35 +0530 IST
      test1/results/99516f5e-b206-4bc5-a6c3-87dc357120f4  2026-01-08 12:26:30 +0530 IST           2026-01-08 12:26:58 +0530 IST
      test1/results/49232b08-3510-4a38-934c-8fb98fdb5245  2026-01-08 12:26:29 +0530 IST           2026-01-08 12:26:47 +0530 IST
      test1/results/d2fd3432-4b10-4a13-b85e-88dffa0c2d29  2026-01-08 12:26:24 +0530 IST           2026-01-08 12:27:35 +0530 IST
      test1/results/a3c19b66-afb4-476a-be05-8917f9a0b257  2026-01-08 12:26:27 +0530 IST           2026-01-08 12:26:39 +0530 IST
      test1/results/a228c871-d6d1-4fb5-a14e-e0319cea60e5  2026-01-08 12:26:27 +0530 IST           2026-01-08 12:26:44 +0530 IST
      test1/results/81c08293-fca6-4447-9fb2-49f7fbf28988  2026-01-08 12:26:26 +0530 IST           2026-01-08 12:27:12 +0530 IST
      test1/results/fdf9e5ba-2848-40e7-81e8-366ae39f3479  2026-01-08 12:26:22 +0530 IST           2026-01-08 12:26:27 +0530 IST
      test1/results/b970c0ac-0700-447a-b217-44bf44ffa026  2026-01-08 12:26:30 +0530 IST           2026-01-08 12:26:47 +0530 IST
      test1/results/c05e3915-2d7c-4ecc-a7e5-d3dec28d3b13  2026-01-08 12:26:29 +0530 IST           2026-01-08 12:26:58 +0530 IST
      test1/results/a8707281-4354-4e2e-a8dd-d15accb9e049  2026-01-08 12:26:24 +0530 IST           2026-01-08 12:27:03 +0530 IST
      test1/results/8a5e015e-fedb-4c16-ba82-b3110c3a4a5c  2026-01-08 12:26:21 +0530 IST           2026-01-08 12:26:41 +0530 IST
      test1/results/caaa9684-5869-4100-912d-6d8ec83e925a  2026-01-08 12:26:20 +0530 IST           2026-01-08 12:27:13 +0530 IST
      test1/results/fb382f1a-5e61-4385-a0b5-068557744a28  2026-01-08 12:26:19 +0530 IST           2026-01-08 12:26:39 +0530 IST
      test1/results/bc8a11bc-85cf-4647-a4ec-e3334e5b9d03  2026-01-08 12:26:31 +0530 IST           2026-01-08 12:26:45 +0530 IST
      test1/results/a37fa8a7-4f52-48ef-a7f8-a8025b13c3f0  2026-01-08 12:26:23 +0530 IST           2026-01-08 12:26:23 +0530 IST
      test1/results/b1bc2da1-6fe2-44fc-83f3-6b21c1368e39  2026-01-08 12:26:29 +0530 IST           2026-01-08 12:27:07 +0530 IST
      test1/results/f4977c86-e19d-4d17-9ccd-b87c5c878dbb  2026-01-08 12:26:25 +0530 IST           2026-01-08 12:26:38 +0530 IST
      test1/results/ed80cae7-2ede-43c9-a7e1-d00010220a79  2026-01-08 12:26:20 +0530 IST           2026-01-08 12:26:24 +0530 IST
      test1/results/1b26e378-ecc3-4a14-b924-89927a78a093  2026-01-08 12:26:25 +0530 IST           2026-01-08 12:26:38 +0530 IST
      test1/results/df751fda-8892-4279-b438-935d4567daac  2026-01-08 12:26:32 +0530 IST           2026-01-08 12:26:47 +0530 IST
      test1/results/70482af9-9d8e-46c5-9a0d-03b5042eb500  2026-01-08 12:26:19 +0530 IST           2026-01-08 12:27:36 +0530 IST
      test1/results/4204b2c8-c5ff-49d4-8141-a011488db87d  2026-01-08 12:26:24 +0530 IST           2026-01-08 12:26:56 +0530 IST
      test1/results/50736a14-421c-4010-a002-f5da38f9c950  2026-01-08 12:26:22 +0530 IST           2026-01-08 12:26:58 +0530 IST
      test1/results/36465a4c-0a48-4c32-ba3c-16af06fd9e83  2026-01-08 12:26:28 +0530 IST           2026-01-08 12:28:07 +0530 IST
      1. Once the results retention time is completed and the current results logs are expired, create new pipelineRun
      $ opc results result list --insecure --addr ${RESULTS_API} test1
      Name                                    Start                                   Update
      [12:32:33] smanthin@smanthin-thinkpadp16vgen1:~/Automation/release-tests$ opc p start caches-python-pipeline 
      1. Wait for the completion of the new pipelineRun 
      $ opc pr ls
      NAME                               STARTED          DURATION   STATUS
      caches-python-pipeline-run-ddtf8   34 seconds ago   ---        Running
      tkn-run-2                          6 minutes ago    16s        Succeeded
      tkn-pac-run-2                      6 minutes ago    17s        Succeeded
      nodejs-ex-git-pr                   7 minutes ago    1m43s      Failed
      pull-request-pipeline-run          7 minutes ago    35s        Failed
      kn-pz-run                          7 minutes ago    19s        Failed
       
      1. While pipelineRun is in progress, validate new results list is appeared with current pipelineRun
      $ opc results result list --insecure --addr ${RESULTS_API} test1
      Name                                                Start                                   Update
      test1/results/d31c7f36-31d9-404e-9a46-c6051aef9157  2026-01-08 12:32:54 +0530 IST           2026-01-08 12:33:26 +0530 IST 

      Actual results:

      Once the new pipelineRun is completed successfully, one more result is being shown in the results list for which retention is already completed

      $ opc results result list --insecure --addr ${RESULTS_API} test1
      Name                                                Start                                   Update
      test1/results/d31c7f36-31d9-404e-9a46-c6051aef9157  2026-01-08 12:32:54 +0530 IST           2026-01-08 12:33:37 +0530 IST
      test1/results/df751fda-8892-4279-b438-935d4567daac  2026-01-08 12:33:38 +0530 IST           2026-01-08 12:33:38 +0530 IST
      tkn-run-2    test1/results/df751fda-8892-4279-b438-935d4567daac 

      When checking the result id, it matches with tkn-run-2   pipelineRun which is already completed and results retention time is already expired for it.

      Expected results:

      Only the current pipelineRun result record should get populated in the results list as the old result records are expired after logRetention time

      Reproducibility (Always/Intermittent/Only Once):

      Occuring when enabling results retention with tektonpruner

      Acceptance criteria: 

       

      Definition of Done:

      Build Details:

      Additional info (Such as Logs, Screenshots, etc):

      PFA logs for reference
      tektonconfig.yaml
      tekton-pruner-controller-logs.log
      tekton-results-api-logs.log
      tekton-results-retention-policy-agent-logs.log

       *

              rh-ee-athorp Andrew Thorp
              rh-ee-smanthin Sai Raju Manthina
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: