vllm

35 progetti condividono questo topic GitHub

Le linee collegano membri che sono misurabilmente correlati tra loro. La dimensione dei punti riflette le stelle.

🧬 Membri

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker…

Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with…

✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field…

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & …

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and…

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU…

Awesome-LLM-Inference

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4,…

semantic-router

A programmable Mixture-of-Models router for heterogeneous LLM inference

Structured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM

learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen

Cascading runtime for AI agents. Optimize cost, latency, quality, and policy decisions inside the agent loop.

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and…

Community maintained hardware plugin for vLLM on Ascend

Open Source Continuous Inference Benchmark Research Platform — Kimi K3 2.8T, MiniMax M3, DeepSeekv4, GLM5 -…

High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX,…

🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get…

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

Multimodal RL training framework for diffusion & omni models

UniRL is a Framework for Unified Multimodal Model Reinforcement Learning

A Python library powered by Language Models (LLMs) for conversational data discovery and analysis.

[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video…

[CVPR 2025] RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete. Official…

Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default,…

super-json-mode

Low latency JSON generation using LLMs ⚡️

Setup and run a local LLM and Chatbot using consumer grade hardware.

Implementation for FP8/INT8 Rollout for RL training without performence drop.

sndr_core_engine

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere…

Agent and LLM API performance monitoring via network packet probe. Measures performance of OpenClaw, Claude,…

Open-source infrastructure for the full inference lifecycle: deploy, observe, scale, optimize, and safely…

Open source Enterprise AI Gateway for API, MCP and agent, and AI model traffic. One Apache-2.0 binary: 72…

🔗 Famiglie affini

inference 74 sglang 9 cuda 78 qwen 74 llm-inference 65 openai-api 61 llm-serving 17 rocm 6

Misurato dai temi di GitHub condivisi da entrambi i progetti, ponderato in base a quanto è raro ciascun tema.