inference-server

10 Projekte teilen dieses GitHub-Topic

Linien verbinden Mitglieder, die messbar miteinander verwandt sind. Die Punktgröße spiegelt die Sterne wider.

🧬 Mitglieder

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu…

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No…

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and…

Turn any computer or edge device into a command center for your computer vision projects.

Open-source inference server and production cluster for all the models your agent needs.

The simplest way to serve AI/ML models in production

An open-source computer vision framework to build and deploy apps in minutes

TurboOCR, >200 img/s OmnidocBench. TensorRT FP16, PP-OCRv6, HTTP + gRPC

Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving,…

🔗 Verwandte Familien

inference 94 apple-silicon 32 mlx 36 gguf 26 huggingface 108 llamacpp 36 llm-inference 76 tensorrt 42

Gemessen anhand der von beiden Projekten geteilten GitHub-Themen, gewichtet nach der Seltenheit jedes Themas.