Can I read BHD25 J on EtoBox?

BHD25 J by Phúc Nhân Nguyễn is a document available to read on EtoBox.

What is BHD25 J about?

The paper presents a comprehensive framework for optimizing scheduling algorithms in Large Language Model (LLM) inference systems, focusing on efficient request routing and scheduling to enhance throughput and minimize latency. It introduces two key schedulers: the Resource-Aware Dynamic (RAD) scheduler for throughput optimality and the SLO-Aware LLM Inference (SLAI) scheduler, which prioritizes requests based on real-time measurements to meet Service Level Objectives (SLOs). Experimental results demonstrat

Author

Phúc Nhân Nguyễn

Language

EN