vllm

48 projetos partilham este topic do GitHub

Linhas conectam membros que estão mensuravelmente relacionados entre si. O tamanho do ponto reflete estrelas.

🧬 Membros

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker…

Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with…

Android in docker solution with noVNC supported, video recording, mcp server and AI-agent

★ 16k · GitHub ↗

✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field…

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & …

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and…

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

semantic-router

An open, programmable decision layer for models and compute.

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU…

A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines

Awesome-LLM-Inference

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4,…

Structured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM

learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen

Cascading runtime for AI agents. Optimize cost, latency, quality, and policy decisions inside the agent loop.

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and…

Community maintained hardware plugin for vLLM on Huawei Ascend

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

★ 2k · GitHub ↗

Community maintained hardware plugin for vLLM on Apple Silicon

★ 1.9k · GitHub ↗

High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX,…

Open Source Continuous Inference Benchmark Research Platform — Kimi K3 2.8T, MiniMax M3, DeepSeekv4, GLM5 -…

StartLux-Decision

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short…

★ 1.6k · GitHub ↗

Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue…

★ 1.5k · GitHub ↗

Multimodal RL training framework for diffusion & omni models

🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get…

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

time-to-first-token

A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization,…

★ 1k · GitHub ↗

Spark-x2.6 open model series. Pushing the Limits of Agentic Capabilities in On-Device Models

★ 860 · GitHub ↗

Halo is an open-source framework built by White Circle for training large language and multimodal models

★ 833 · GitHub ↗

UniRL is a Framework for Unified Multimodal Model Reinforcement Learning

A Python library powered by Language Models (LLMs) for conversational data discovery and analysis.

[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video…

[CVPR 2025] RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete. Official…

本地优先的全栈 AI 创作工作站：AI对话 · AI漫画 · 漫剧创作 · 写作台 ·…

★ 529 · GitHub ↗

Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default,…

super-json-mode

Low latency JSON generation using LLMs ⚡️

Setup and run a local LLM and Chatbot using consumer grade hardware.

Implementation for FP8/INT8 Rollout for RL training without performence drop.

llm-inference-capacity-handbook

大模型推理案头手册（开源版）：给定 GPU 算力，一个模型能扛多少 QPS？三层模型…

★ 185 · GitHub ↗

deepseek-local-guide-ru

🐋 Полный гайд на русском: как установить и запустить DeepSeek…

★ 157 · GitHub ↗

sndr_core_engine

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere…

Agent and LLM API performance monitoring via network packet probe. Measures performance of OpenClaw, Claude,…

glm-5.3-flash-2x-rtx-pro-6000-blackwell

Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive…

★ 61 · GitHub ↗

Open-source infrastructure for the full inference lifecycle: deploy, observe, scale, optimize, and safely…

global-capacity-orchestrator-on-aws

AI/ML and HPC at planetary scale. One API. Every Accelerator. Any Region. MIT-0 licensed

★ 52 · GitHub ↗

Open source Enterprise AI Gateway for API, MCP and agent, and AI model traffic. One Apache-2.0 binary: 72…

🔗 Familias relacionadas

sglang 15 inference 81 llm-inference 80 cuda 82 qwen 87 local-llm 60 llama-cpp 33 mlx 39

Medido a partir dos tópicos do GitHub compartilhados por ambos os projetos, ponderado pela raridade de cada tópico.