inference-engine
18 progetti condividono questo topic GitHub
Le linee collegano membri che sono misurabilmente correlati tra loro. La dimensione dei punti riflette le stelle.
🧬 Membri
Halfrost-Field
✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field…
★ 13.3k
kimi-k3-in-c
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS,…
★ 9.2k
FedML
FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and…
★ 4.1k
kvcached
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
★ 2k · GitHub ↗
sonar
Large-scale LLM inference engine
★ 1.9k
nobodywho
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
★ 1.6k · GitHub ↗
zero-to-sglang
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch,…
★ 1.6k · GitHub ↗
xllm
A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.…
★ 1.6k
halogen-flash-server
Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models,…
★ 1.1k · GitHub ↗
mlxstudio
MLX Studio - Easiest way to run LLM's on your Mac. All in one engine.
★ 977
ZhiLight
A highly optimized LLM inference acceleration engine for Llama and its variants.
★ 905
TensorSharp
A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console…
★ 632
krasis
Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM…
★ 522
OpenArc
Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over…
★ 486
MoE-Infinity
PyTorch library for cost-effective, fast and easy serving of MoE models.
★ 363
Locals
Run many AI agents at once on your Mac, up to 3x faster.
★ 356 · GitHub ↗
swiftLLM
A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with…
★ 329
edge-dit.cpp
edge-dit.cpp — a native C/C++ inference engine for Diffusion Transformers (DiT), designed for local and…
★ 54 · GitHub ↗
🔗 Famiglie affini
Misurato dai temi di GitHub condivisi da entrambi i progetti, ponderato in base a quanto è raro ciascun tema.