cronjob for a namespace is not getting deleted after disabling pruner from config CR

Steps to Reproduce

1. Edit config CR with below values for pruner part 

   pruner:
       keep: 2
       resources:
         - taskrun
         - pipelinerun
       schedule: '*/1 * * * *'
   

2. Add annotation operator.tekton.dev/prune.schedule: "/2 * * * * "
3. Disable pruner in config CR

Expected Behaviour
Cronjob created in the namespace should get deleted

Actual Behaviour
Cronjob created in the namespace is not getting deleted

Note: Cronjob is not getting delete even after removing the operator.tekton.dev/prune.schedule: "*/2 * * * *" annotation