llm-serving

23 progetti condividono questo topic GitHub

Le linee collegano membri che sono misurabilmente correlati tra loro. La dimensione dei punti riflette le stelle.

🧬 Membri

A high-throughput and memory-efficient inference and serving engine for LLMs

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for…

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and…

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer,…

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model…

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU…

Superduper: End-to-end framework for building custom AI applications and agents.

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

High-performance inference framework for large language models, focusing on efficiency, flexibility, and…

Community maintained hardware plugin for vLLM on Ascend

MoBA: Mixture of Block Attention for Long-Context LLMs

AICI: Prompts as (Wasm) Programs

Parallax is a distributed model serving framework that lets you build your own AI cluster anywhere

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

A highly optimized LLM inference acceleration engine for Llama and its variants.

♾️ Private Agent Fleet with Spec Coding. Each agent gets their own GPU-accelerated desktop. Run Claude,…

LLM-FineTuning-Large-Language-Models

LLM (Large Language Model) FineTuning

Efficient AI Inference & Serving

A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with…

sndr_core_engine

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere…

🔗 Famiglie affini

llm-inference 76 inference 94 model-serving 26 cuda 90 mlops 103 qwen 102 llm-training 20 vllm 47

Misurato dai temi di GitHub condivisi da entrambi i progetti, ponderato in base a quanto è raro ciascun tema.