Aphrodite Engine
High-throughput LLM serving
Inference serverInference serverAGPL-3.0Self-hostedPythonAdvanced
A high count usually means the project is heavily examined. A low count usually means it is little examined — not that it is safe.
No public activity signals for this tool — it has no open repository we can measure.
🐳 Docker pulls
📦 PyPI downloads / month
📦 npm downloads / month
🚀 Latest release ·
What is Aphrodite Engine?
Aphrodite Engine is a high-throughput inference server based on vLLM, optimized for serving many users at once with broad quantization and sampling support.
Why people choose Aphrodite Engine
- Very high throughput serving
- Wide quantization support
- Rich sampling options
Aphrodite Engine at a glance
| Category | Inference server |
| Type | Inference server |
| License | AGPL-3.0 |
| Runs locally | Self-hosted |
| Built with | Python |
| Skill level | Advanced |
| Best for | serving many users at high throughput |
Latest releases
Open-source alternatives to Aphrodite Engine
Other open-source inference server tools worth comparing:
vLLMHigh-throughput serving for production
TGIHugging Face's production text server
SGLangFast serving with structured outputs
LMDeployToolkit for compressing and serving LLMs
TensorRT-LLMPeak throughput on NVIDIA GPUs
OpenLLMServe any open model as an OpenAI API in one command
KTransformersRun huge MoE models on one consumer GPU
Ray ServeScale model serving across a clusterBentoMLPackage any model into a production APIAphrodite Engine head-to-head
Aphrodite Engine vs vLLMAphrodite Engine vs TGIAphrodite Engine vs SGLangAphrodite Engine vs LMDeployAphrodite Engine vs TensorRT-LLMAphrodite Engine vs OpenLLMAphrodite Engine vs KTransformersAphrodite Engine vs Ray ServeAphrodite Engine vs BentoML
FAQ
Is Aphrodite Engine free?
Aphrodite Engine is free and open-source (AGPL-3.0 license), so you can use, self-host and modify it at no cost.
Can I run Aphrodite Engine locally?
Yes. Aphrodite Engine is designed to run on your own machine or server, keeping your data private.
What is the best alternative to Aphrodite Engine?
Popular open-source alternatives include vLLM, TGI, SGLang. See the comparisons above to choose.
Explore more open-source AI
Browse the full directory of open-source AI tools, models and projects — updated daily.
Browse all tools →