The finally tasks do not get removed and remain in the pipeline.    

    In all supported OCP version

    Always

1. Create a finally task in a pipeline in pipeline builder
2. Save pipeline
3. Edit pipeline and remove finally task in pipeline builder
4. Save pipeline
5. Observe that the finally task has not been removed

The finally tasks do not get removed and remain in the pipeline.    

Finally task gets removed from pipeline when removing the finally tasks and saving the pipeline in the "pipeline builder" mode.