The DSPA deployment has images parameterized via config, all the images of the different components are parametrized and are used directly during the deployment.
https://github.com/opendatahub-io/data-science-pipelines-operator/blob/main/config/configmaps/files/config.yaml 

we would like to extend the same behavior for the oauth-proxy image as well.

Reference: https://issues.redhat.com/browse/RHODS-8604

Ensure that the images are passed in via odh-deployer like all the other images in the kfdef: https://github.com/red-hat-data-services/odh-deployer/blob/main/kfdefs/rhods-data-science-pipelines-operator.yaml (note how mariadb & moveresultsimage are hardcoded, oauth proxy should be similar).