llama.cpp

The C/C++ engine powering local inference

Run LLMs locallyInference library (C/C++)MITRuns locallyC/C++Advanced

🛡 Published security advisories

A high count usually means the project is heavily examined. A low count usually means it is little examined — not that it is safe.

No public activity signals for this tool — it has no open repository we can measure.

OSAI Pulse ⓘ ★★★★★★★★★★ /100 signals tracked

🐳 Docker pulls 📦 PyPI downloads / month 📦 npm downloads / month 🚀 Latest release ·

Visit website →View on GitHub

What is llama.cpp?

llama.cpp is the high-performance C/C++ inference engine that underpins most local LLM tools, supporting GGUF models with aggressive quantization across CPUs and GPUs.

Why people choose llama.cpp

Runs almost anywhere, from laptops to Raspberry Pi
State-of-the-art quantization (GGUF) for tiny footprints
The engine many other tools are built on top of

llama.cpp at a glance

Category	Run LLMs locally
Type	Inference library (C/C++)
License	MIT
Runs locally	Yes
Built with	C/C++
Skill level	Advanced
Best for	developers who want maximum control and portability

Open-source alternatives to llama.cpp

Other open-source run llms locally tools worth comparing:

OllamaRun open LLMs locally from one command

LM StudioBrowse, download and chat with local models

JanOpen-source, offline ChatGPT-style desktop app

GPT4AllPrivate local AI that runs on CPU

LocalAIA drop-in OpenAI API you self-host

Text Generation WebUIFeature-rich web UI for local models

KoboldCppSingle-file local model runner

MLC LLMRun LLMs on any device, even phones

llamafileOne executable file = model + runtime

exoRun big models across your everyday devices

CortexOllama-style runtime from the Jan team

Nexa SDKRun any model on any device — CPU, GPU, NPU

RamaLamaRun models as OCI containers

GPUStackManage GPU clusters for running models

llama.cpp head-to-head

llama.cpp vs Ollama llama.cpp vs LM Studio llama.cpp vs Jan llama.cpp vs GPT4All llama.cpp vs LocalAI llama.cpp vs Text Generation WebUI llama.cpp vs KoboldCpp llama.cpp vs MLC LLM llama.cpp vs llamafile llama.cpp vs exo llama.cpp vs Cortex llama.cpp vs Nexa SDK llama.cpp vs RamaLama llama.cpp vs GPUStack

FAQ

Is llama.cpp free?

llama.cpp is free and open-source (MIT license), so you can use, self-host and modify it at no cost.

Can I run llama.cpp locally?

Yes. llama.cpp is designed to run on your own machine or server, keeping your data private.

What is the best alternative to llama.cpp?

Popular open-source alternatives include Ollama, LM Studio, Jan. See the comparisons above to choose.

Explore more open-source AI

Browse the full directory of open-source AI tools, models and projects — updated daily.

Browse all tools →