pretraining

15 projects share this GitHub topic

Lines connect members that are measurably related to each other. Dot size reflects stars.

🧬 Members

LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

★ 107.4k

Llama-Chinese

★ 14.8k

LMOps

General technology for enabling AI capabilities w/ LLMs and MLLMs

★ 4.5k

mPLUG-Owl

mPLUG-Owl: The Powerful Multi-modal Large Language Model Family

★ 2.5k

HRM-Text

HRM-Text is a 1B text generation model based on the HRM architecture, strengthened by task completion and…

★ 2.2k

MARS

The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models

★ 725

Craw4LLM

Official repository for "Craw4LLM: Efficient Web Crawling for LLM Pretraining"

★ 666

TPA

[NeurIPS 2025 Spotlight] TPA: Tensor ProducT ATTenTion Transformer (https://arxiv.org/abs/2501.06425)

★ 464

bigdetection

BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-training

★ 399

VL-T5

PyTorch code for "Unifying Vision-and-Language Tasks via Text Generation" (ICML 2021)

★ 372

data_management_LLM

Collection of training data management explorations for large language models

★ 343

LLMInternSkill

LLMInternSkill: LLM internship resume and job-search Codex Skill for resume polish, JD tailoring, evidence…

★ 333

ChatPLUG

A Chinese Open-Domain Dialogue System

★ 323

hands-on-llm

动手学大模型全栈：CS336 中文精讲 · PyTorch 手搓 Transformer · 单卡复现…

★ 119 · GitHub ↗

awesome-vintage-llms

A curated list of vintage large language models — also called historical or time-capsule LLMs — trained…

★ 45 · GitHub ↗

Measured from GitHub topics shared by both projects, weighted by how rare each topic is.