vllm

47 projets partagent ce topic GitHub

Les traits relient les membres réellement apparentés entre eux. La taille des points suit les étoiles.

🧬 Membres

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker…

Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with…

✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field…

AI-Research-SKILLs

Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills…

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & …

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and…

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework…

A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines

Awesome-LLM-Inference

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4,…

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU…

Structured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM

Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc

semantic-router

Intelligent Mixture-of-Models Router for Efficient Heterogeneous LLMs Inference

A course of learning LLM inference serving on Apple Silicon for systems engineers: build a tiny vLLM + Qwen.

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

Cascading runtime for AI agents. Optimize cost, latency, quality, and policy decisions inside the agent loop.

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and…

Community maintained hardware plugin for vLLM on Ascend

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA,…

OpenAI and Anthropic compatible server for Apple Silicon. Run LLMs and vision-language models (Llama,…

Open Source Continuous Inference Benchmark Research Platform — Kimi K3 2.8T, MiniMax M3, DeepSeekv4, GLM5 -…

🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get…

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and…

prometheus-eval

Evaluate your LLM's response with Prometheus and GPT4 💯

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

UniRL is a Framework for Unified Multimodal Model Reinforcement Learning

Make Discord your LLM frontend - Supports any OpenAI compatible API (OpenRouter, Ollama and more)

A Python library powered by Language Models (LLMs) for conversational data discovery and analysis.

[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video…

Multimodal RL training framework for diffusion & omni models

Accurate, large-scale, and extensible simulator for LLM inference Systems

EfficientSAM3 compresses SAM3 into lightweight, edge-friendly models via progressive knowledge distillation…

[CVPR 2025] RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete. Official…

Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default,…

The open creative AI workspace

DeepSeek-OCR-WebUI

🎨 Ready-to-use DeepSeek-OCR Web UI | Modern Interface | 7 Recognition Modes | Batch Processing | …

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM,…

TopicGPT: A Prompt-Based Framework for Topic Modeling [NAACL'24]

super-json-mode

Low latency JSON generation using LLMs ⚡️

Setup and run a local LLM and Chatbot using consumer grade hardware.

Implementation for FP8/INT8 Rollout for RL training without performence drop.

sndr_core_engine

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere…

Agent and LLM API performance monitoring via network packet probe. Measures performance of OpenClaw, Claude,…

★ 66 · GitHub ↗

Self-hosted AI gateway and LLM proxy. OpenAI-compatible API for OpenAI, Anthropic, Gemini, Bedrock and 60+…

★ 47 · GitHub ↗

🔗 Familles voisines

inference 94 sglang 11 cuda 90 qwen 102 llm-inference 76 mistral 66 huggingface-transformers 18 llm-serving 23

Mesuré à partir des topics GitHub communs aux deux projets, pondérés par leur rareté.