quantization

51 projetos partilham este topic do GitHub

Linhas conectam membros que estão mensuravelmente relacionados entre si. O tamanho do ponto reflete estrelas.

🧬 Membros

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

Faster Whisper transcription with CTranslate2

Chinese-LLaMA-Alpaca

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

[🔥updating ...] AI 自动量化交易机器人(完全本地部署) AI-powered Quantitative Investment…

A vector index built on TurboQuant, written in Rust with Python bindings

Accessible large language models via k-bit quantization for PyTorch.

Fast inference engine for Transformer models

[ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to…

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with…

Pretrained-Language-Model

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression…

AIMET is a library that provides advanced quantization and compression techniques for trained neural network…

Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an…

Awesome-Model-Quantization

A list of papers, docs, codes about model quantization. This repo is aimed to provide the info for model…

AI-Engineering.academy

Mastering Applied AI, One Concept at a Time

mixtral-offloading

Run Mixtral-8x7B models in Colab or consumer desktops

ai-engineering-interview-questions

Your Cheat Sheet for AI Engineering Interview – Questions and Answers.

PPL Quantization Tool (PPQ) is a powerful offline neural network quantization tool.

Run a 1-billion parameter LLM on a $10 board with 256MB RAM

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

model-optimization

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

Brevitas: neural network quantization in PyTorch

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA,…

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.

Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

Build computer vision models in a fraction of the time and with less data.

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and…

Neural Network Compression Framework for enhanced OpenVINO™ inference

Z80-μLM is a 2-bit quantized language model small enough to run on an 8-bit Z80 processor. Train…

A powerful model compression framework for LLMs and LVLMs, adapted for NVIDIA GPUs and Huawei Ascend NPUs.

TinyChatEngine: On-Device LLM Inference Library

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

Awesome-Quantization-Papers

List of papers related to neural network quantization in recent AI conferences and journals.

[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video…

[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

hailo_model_zoo

The Hailo Model Zoo includes pre-trained models and a full building and evaluation environment

Welcome to the official repository of SINQ! A novel, fast and high-quality quantization method designed to…

Offline semantic Text-to-Image and Image-to-Image search on Android powered by quantized state-of-the-art…

Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)

[ICLR 2026]QeRL enables RL for 32B LLMs on a single H100 GPU.

[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models(NeurIPS 2024…

[ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

quick-start-guide-to-llms

The Official Repo for "Quick Start Guide to Large Language Models"

[ICCV 2023] Q-Diffusion: Quantizing Diffusion Models.

Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache…

BitNet-Transformers

0️⃣1️⃣🤗 BitNet-Transformers: Huggingface Transformers Implementation of "BitNet: Scaling 1-bit…

On-device LLM Inference Powered by X-Bit Quantization

PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving,…

sndr_core_engine

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere…

🔗 Familias relacionadas

model-compression 19 inference 94 pruning 9 compression 22 lora 65 qwen 102 peft 19 cuda 90

Medido a partir dos tópicos do GitHub compartilhados por ambos os projetos, ponderado pela raridade de cada tópico.