gguf

27 projects share this GitHub topic

Lines connect members that are measurably related to each other. Dot size reflects stars.

🧬 Members

Hundreds of models & providers. One command to find what runs on your hardware.

Distribute and run LLMs with a single file.

KoboldCpp - Run GGUF AI models locally with a KoboldAI UI. One File. Zero Install.

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No…

Atomic Agent is a local-first AI agent. Runs open-weight models on your own machine via llama.cpp.

The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs,…

动手学Ollama，CPU玩转大模型部署，在线阅读地址：https://datawhalechina.github.io/handy-ollam…

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model…

Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend,…

Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your…

Go manage your Ollama models

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

★ 1.6k · GitHub ↗

StartLux-Decision

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short…

★ 1.6k · GitHub ↗

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA,…

Interface for OuteTTS models.

overlay-translator

The most advanced, fully offline client-side AI suite on Android today.

Llama 3+ inference in pure Java

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console…

A desktop app for running Large Language Models locally.

deepseek-local-guide-ru

🐋 Полный гайд на русском: как установить и запустить DeepSeek…

★ 157 · GitHub ↗

High-performance Lemonade alternative for AMD Strix Halo & Radeon — latest MoE models (Ling-3.0-Flash, Qwen…

★ 94 · GitHub ↗

llama.cpp optimized for NVIDIA Tesla V100 (SM70): Qwen3.8-Flash-Next with MTP (prefill 3402 tok/s at 200K…

★ 59 · GitHub ↗

Private, on-device AI desktop app — GGUF (llama.cpp) & MLX, running Muse-Glimmer and Qwen3.8 with their…

★ 48 · GitHub ↗

Agentic Runtime

★ 42 · GitHub ↗

Native AI inference for PHP 8.3+ - run ONNX, GGUF (llama.cpp) and RubixML models directly in your PHP process…

🔗 Related families

local-llm 60 llama-cpp 33 llm-inference 80 mlx 39 local-ai 58 inference 81 qwen 87 gemma 29

Measured from GitHub topics shared by both projects, weighted by how rare each topic is.