Details
-
Story
-
Resolution: Unresolved
-
Normal
-
None
-
None
-
False
-
-
False
-
RHOAISTRAT-93 - Model serving 1H24 enhancements
Description
Feature description
As it right now, we are creating a Route object for every InferenceService instance we deploy in a Project. That can lead to serious performance issues.
We are working with the model serving team to adapt the controller for this specific scenario:
- When a Serving Runtime is created in a namespace, the odh-model-controller will create a route with a deterministic name based on the namespace (project)
- Each time we create a new inference service, it will add a path with a deterministic format if external route is enabled.
- This logic will be available for both inferenceSErvices already deployed and new ones.
- For existing inference services, the old route will be available in case it's being used for production.