quantization

36 progetti condividono questo topic GitHub

Le linee collegano membri che sono misurabilmente correlati tra loro. La dimensione dei punti riflette le stelle.

🧬 Membri

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

Faster Whisper transcription with CTranslate2

Chinese-LLaMA-Alpaca

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

A vector index built on TurboQuant, written in Rust with Python bindings

Accessible large language models via k-bit quantization for PyTorch.

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS,…

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to…

Pretrained-Language-Model

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

ai-engineering-interview-questions

Your Cheat Sheet for AI Engineering Interview – Questions and Answers.

neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression…

Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an…

AI-Engineering.academy

Mastering Applied AI, One Concept at a Time

mixtral-offloading

Run Mixtral-8x7B models in Colab or consumer desktops

Run a 1-billion parameter LLM on a $10 board with 256MB RAM

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and…

A powerful model compression framework for LLMs and LVLMs, adapted for NVIDIA GPUs and Huawei Ascend NPUs.

TinyChatEngine: On-Device LLM Inference Library

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

Awesome-Quantization-Papers

List of papers related to neural network quantization in recent AI conferences and journals.

[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video…

[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

Welcome to the official repository of SINQ! A novel, fast and high-quality quantization method designed to…

[ICLR 2026]QeRL enables RL for 32B LLMs on a single H100 GPU.

[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

[ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models(NeurIPS 2024…

quick-start-guide-to-llms

The Official Repo for "Quick Start Guide to Large Language Models"

Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache…

On-device LLM Inference Powered by X-Bit Quantization

BitNet-Transformers

0️⃣1️⃣🤗 BitNet-Transformers: Huggingface Transformers Implementation of "BitNet: Scaling 1-bit…

sndr_core_engine

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere…

A native macOS lab for teaching tiny language models to think — build the architecture, train the weights,…

Run Ornith, Qwen, Nemotron & DeepSeek optimized on AMD Strix Halo — unified OpenAI-compatible server with…

★ 41 · GitHub ↗

🔗 Famiglie affini

qwen 74 model-compression 10 language-model 113 lora 47 peft 16 inference 74 llm-inference 65 transformer 59

Misurato dai temi di GitHub condivisi da entrambi i progetti, ponderato in base a quanto è raro ciascun tema.