inference-engine

12 projetos partilham este topic do GitHub

Linhas conectam membros que estão mensuravelmente relacionados entre si. O tamanho do ponto reflete estrelas.

🧬 Membros

✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field…

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS,…

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and…

Large-scale LLM inference engine

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.…

MLX Studio - Home of JANG_Q - Image Gen/Edit + Chat/Code All in one - + OpenClaw (Anthropic API)

A highly optimized LLM inference acceleration engine for Llama and its variants.

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM…

Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over…

A native .NET LLM inference engine for GGUF models. TensorSharp provides a console application, a web-based…

PyTorch library for cost-effective, fast and easy serving of MoE models.

A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with…

🔗 Familias relacionadas

llm-inference 65 cuda 78 inference 74 mixture-of-experts 12 transformer 59 glm 25 qwen 74 algorithms 43

Medido a partir dos tópicos do GitHub compartilhados por ambos os projetos, ponderado pela raridade de cada tópico.