quantization
41 Projekte teilen dieses GitHub-Topic
Linien verbinden Mitglieder, die messbar miteinander verwandt sind. Die Punktgröße spiegelt die Sterne wider.
🧬 Mitglieder
LlamaFactory
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
★ 75.7k
faster-whisper
Faster Whisper transcription with CTranslate2
★ 24.3k
Chinese-LLaMA-Alpaca
中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)
★ 18.9k
turbovec
A vector index built on TurboQuant, written in Rust with Python bindings
★ 17.4k
kimi-k3-in-c
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS,…
★ 9.2k
bitsandbytes
Accessible large language models via k-bit quantization for PyTorch.
★ 8.5k
SageAttention
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to…
★ 4k
ai-engineering-interview-questions
Your Cheat Sheet for AI Engineering Interview – Questions and Answers.
★ 3.6k
Pretrained-Language-Model
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
★ 3.2k
neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression…
★ 2.7k
xTuring
Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an…
★ 2.7k
AI-Engineering.academy
Mastering Applied AI, One Concept at a Time
★ 2.4k
mixtral-offloading
Run Mixtral-8x7B models in Colab or consumer desktops
★ 2.3k
picolm
Run a 1-billion parameter LLM on a $10 board with 256MB RAM
★ 2.2k
project-maya
Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B…
★ 2.1k · GitHub ↗
AngelSlim
Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.
★ 1.8k
rwkv.cpp
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
★ 1.6k
gpu_poor
Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization
★ 1.4k
GPTQModel
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and…
★ 1.2k
MindPipe
A powerful model compression framework for LLMs and LVLMs, adapted for NVIDIA GPUs and Huawei Ascend NPUs.
★ 1k
TinyChatEngine
TinyChatEngine: On-Device LLM Inference Library
★ 970
OmniQuant
[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.
★ 964
Awesome-Quantization-Papers
List of papers related to neural network quantization in recent AI conferences and journals.
★ 855
halo
Halo is an open-source framework built by White Circle for training large language and multimodal models
★ 833 · GitHub ↗
LightCompress
[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video…
★ 751
SqueezeLLM
[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization
★ 725
SINQ
Welcome to the official repository of SINQ! A novel, fast and high-quality quantization method designed to…
★ 635
QeRL
[ICLR 2026]QeRL enables RL for 32B LLMs on a single H100 GPU.
★ 511
KVQuant
[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
★ 437
KIVI
[ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
★ 435
PiSSA
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models(NeurIPS 2024…
★ 430
quick-start-guide-to-llms
The Official Repo for "Quick Start Guide to Large Language Models"
★ 395
KVSplit
Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache…
★ 362
picollm
On-device LLM Inference Powered by X-Bit Quantization
★ 317
BitNet-Transformers
0️⃣1️⃣🤗 BitNet-Transformers: Huggingface Transformers Implementation of "BitNet: Scaling 1-bit…
★ 316
deepseek-local-guide-ru
🐋 Полный гайд на русском: как установить и запустить DeepSeek…
★ 157 · GitHub ↗
sndr_core_engine
SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere…
★ 133
QuantClaw-plugin
QuantClaw is a plug-and-play task-type routing quantization plugin for OpenClaw.
★ 115 · GitHub ↗
halofpx
High-performance Lemonade alternative for AMD Strix Halo & Radeon — latest MoE models (Ling-3.0-Flash, Qwen…
★ 94 · GitHub ↗
edge-dit.cpp
edge-dit.cpp — a native C/C++ inference engine for Diffusion Transformers (DiT), designed for local and…
★ 54 · GitHub ↗
LabLLM
A native macOS lab for teaching tiny language models to think — build the architecture, train the weights,…
★ 42
🔗 Verwandte Familien
Gemessen anhand der von beiden Projekten geteilten GitHub-Themen, gewichtet nach der Seltenheit jedes Themas.