Uploaded image for project: 'AI Platform Core Components'
  1. AI Platform Core Components
  2. AIPCC-2104

Enable Same-Node Benchmarking in Model Benchmark

    • Icon: Initiative Initiative
    • Resolution: Duplicate
    • Icon: Undefined Undefined
    • None
    • None
    • Model Validation
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      Goal:

      Allow benchmarking scenarios where both the client and the model server run on the same physical node.

       

      Note:{}

      While Jounce is implementing this feature at the request of the PSAP team, we do not recommend using same-node benchmarking as a general practice.

       

      Why:{}

      In real-world production environments, the client and the model server are almost never colocated on the same physical node. Benchmarking under these artificial conditions might yield lower latency numbers, but they are misleading and do not reflect actual deployment performance.

      This setup can give a false sense of optimization and lead to conclusions that won’t hold in realistic, distributed systems—where network latency, node placement, and load balancing all play a significant role in true performance.

      Scope:{}

      • Add support for colocated client and server execution within the same Kubernetes node.
      • Update JBenchmark infrastructure to schedule and bind both components to the same node.
      • Ensure compatibility with existing benchmarking workflows and runtime environments.

       

      Outcome:{}

      More reliable latency and throughput measurements, particularly useful for local benchmarks, regression testing, and fine-grained performance tuning.

              rh-ee-abadli Aviran Badli (Inactive)
              rh-ee-abadli Aviran Badli (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: