model-serving

26 projets partagent ce topic GitHub

Les traits relient les membres réellement apparentés entre eux. La taille des points suit les étoiles.

🧬 Membres

A high-throughput and memory-efficient inference and serving engine for LLMs

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model…

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework…

A framework for efficient model inference with omni-modality models

Open-Source Personal Cloud OS for Always-On Agents

Deep-Learning-in-Production

In this repository, I will share some useful notes and references about deploying deep learning-based models…

AI-Infra-from-Zero-to-Hero

🚀 Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice. ⚡️ System for…

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its…

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and…

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

High-performance inference framework for large language models, focusing on efficiency, flexibility, and…

Community maintained hardware plugin for vLLM on Ascend

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

🏕️ Reproducible development environment for humans and agents

AICI: Prompts as (Wasm) Programs

MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across…

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

The simplest way to serve AI/ML models in production

A highly optimized LLM inference acceleration engine for Llama and its variants.

Serverless LLM Serving for Everyone.

Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs…

The missing bridge between your ML models and your AI agents.

BentoDiffusion: A collection of diffusion models served with BentoML

A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with…

dgx-spark-inference-stack

Serve the home! Inference stack for your Nvidia DGX Spark aka the Grace Blackwell AI supercomputer on your…

★ 50 · GitHub ↗

🔗 Familles voisines

inference 94 llm-serving 23 mlops 103 llm-inference 76 cuda 90 inference-engine 14 gpu 93 blackwell 6

Mesuré à partir des topics GitHub communs aux deux projets, pondérés par leur rareté.