vLLM

High-throughput open-source engine for serving models on your own GPUs

vLLM is a fast, open-source inference engine that lets teams serve open models on their own GPUs, keeping data in-house while scaling to many users. Built for: teams self-hosting models. Format: Python library + server. License: Apache-2.0.