To deliver RHODS in Serverless mode,  i.e to provide capabilities such as canary deployments, A/B testing, and autoscaling up and down. 

One solution is to embrace the upstream Kserve platform that uses Knative to provide:

• Modern serverless inference workload with Autoscaling including Scale to Zero on GPU
• Advanced deployments with canary rollout

In order to support RHODS service for Kserve, Serverless team needs to provide Hybrid SRE support through Engineers, any integration work that might be required