Uploaded image for project: 'AI Platform Core Components'
  1. AI Platform Core Components
  2. AIPCC-5844

Package ITS Hub (Inference Time Scaling) Dependencies to Red Hat Maintained Python Index

    • Package ITS Hub dependencies to Red Hat Maintained Python Index
    • False
    • Hide

      None

      Show
      None
    • False
    • In Progress
    • 20% To Do, 20% In Progress, 60% Done

       

      Requested Package Name and Version:

      its_hub: github.com/Red-Hat-AI-Innovation-Team/its_hub

      dependencies = [
          "openai>=1.75.0",
          "tqdm>=4.65.0",
          "typing-extensions>=4.0.0",
          "reward-hub==0.1.2",
          "transformers==4.53.2",  # Pin to exact version that worked in CI to avoid aimv2 config conflict with vLLM 0.9.1
          "backoff>=2.2.0",
          "click>=8.1.0",
          "fastapi>=0.115.0",
          "uvicorn>=0.34.0",
          "pydantic>=2.11.0",
          "numpy>=1.24.0",
          "requests>=2.28.0",
          "aiohttp>=3.8.0"
      ]

      Brief Explanation for request

      Enable Red Hat AI and OpenShift AI customers to build custom inference images by installing its_hub and its dependencies straight from the Red Hat internal Python index. So Red Hat AI and OpenShift AI customers can pull ITS algorithms directly via pip to build their own custom inference images. 

      ITS Hub focuses on accuracy-boosting at inference time—without retraining. The library integrates with vLLM through an OpenAI-compatible interface.

      https://github.com/Red-Hat-AI-Innovation-Team/its_hub/releases/tag/v0.2.4

      • Reduce non-required dependency (accelerate), and upgrade core dependencies (vllm >=0.10.0). 
      • Updated feature of offering inference-time-scaling as an API endpoint.
      • Updated notebook for running self-consistency algorithm.

       

      QE user acceptance tests

      Package enables customers to be able to use its_hubs Inference-Time scaling algorithms. 

      Package License

      Apache 2.0 https://github.com/Red-Hat-AI-Innovation-Team/its_hub/blob/main/LICENSE

       

              Unassigned Unassigned
              rh-ee-linglis Luke Inglis
              Doug Hellmann, GX Xu, Kai Xu
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: