Home› Projects› flashinfer
flashinfer
Python

flashinfer

FlashInfer: Kernel Library for LLM Serving

by flashinfer-ai · GitHub
6.3k
Stars
1.4k
Forks
+117 this week
Trending
Apache-2.0
License
2023
Created
today
Last commit
attention
Category
Python
Language
attentioncudadistributed-inferenceApache-2.0Python
View on GitHub
In plain words

Use optimized code to improve the performance of AI models on various graphics cards during inference.

You maintain this project?

Claim its page: indexed whatever its rank, translated into six languages, and enriched with what you write yourself.

Claim this page →
flashinfer — GitHub preview card
📈 Star history
6.3k5.9k
2026-07-122026-08-31
📈 Track flashinfer

Get an email alert on its next release or when it starts trending — never miss the moment.

Free · no card · unsubscribe anytime
Get email alerts →
📄 About

FlashInfer: Kernel Library for LLM Serving

flashinfer has 6.3k stars on GitHub. It has been forked 1.4k times. flashinfer is written mainly in Python. It has been in active development since 2023. flashinfer is available under the Apache-2.0 license. Its main topics are attention, cuda, distributed-inference, gpu.

❓ Frequently asked questions

What is flashinfer?

FlashInfer: Kernel Library for LLM Serving

Is flashinfer open source?

flashinfer is an open-source project. It is released under the Apache-2.0 license.

Is flashinfer free?

Yes. flashinfer is free and open source — you can use, modify and self-host it.

What license does flashinfer use?

flashinfer is available under the Apache-2.0 license.

What language is flashinfer written in?

flashinfer is written mainly in Python.

🏅 Maintainer of this project?
olud.ai badge — flashinfer

Add this live badge to your README — your GitHub stars and directory rank, refreshed daily.

[![olud.ai](https://olud.ai/badge.php?tool=flashinfer-ai-flashinfer)](https://olud.ai/project/flashinfer-ai-flashinfer.html)
More badge options →
🧬 Shares DNA with🧬 View the DNA map →
torchrec
Pytorch domain library for recommendation systems
★ 2.6k · cuda
sharesgpucuda
TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLM…
★ 14.5k · blackwell
sharesmoecuda
uccl
UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache…
★ 1.5k · ai
sharesmoenvidia
isaac_ros_common
Common utilities, packages, scripts, and testing infrastructure for Isaac ROS packages.
★ 307 · gpu
sharesnvidiagpu
isaac_ros_nvblox
NVIDIA-accelerated 3D scene reconstruction and Nav2 local costmap provider using nvblox
★ 724 · 3d-reconstruction
sharesnvidiagpu
vins-application
VINS-Fusion, VINS-Fisheye, OpenVINS, EnVIO, ROVIO, S-MSCKF, ORB-SLAM2, NVIDIA Elbrus applicatio…
★ 607 · cuda
sharesnvidiacuda
annotated_deep_learning_paper_implementations
🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including…
★ 67.1k · attention
sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
★ 32.9k · attention
numpy-ml
Machine learning, in numpy
★ 16.3k · attention
OpenMythos
A theoretical reconstruction of the Claude Mythos architecture, built from first principles usi…
★ 14.7k · ai

Measured from GitHub topics shared by both projects, weighted by how rare each topic is.