pretraining

13 progetti condividono questo topic GitHub

Le linee collegano membri che sono misurabilmente correlati tra loro. La dimensione dei punti riflette le stelle.

🧬 Membri

LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

General technology for enabling AI capabilities w/ LLMs and MLLMs

mPLUG-Owl: The Powerful Multi-modal Large Language Model Family

HRM-Text is a 1B text generation model based on the HRM architecture, strengthened by task completion and…

The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models

Official repository for "Craw4LLM: Efficient Web Crawling for LLM Pretraining"

[NeurIPS 2025 Spotlight] TPA: Tensor ProducT ATTenTion Transformer (https://arxiv.org/abs/2501.06425)

BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-training

PyTorch code for "Unifying Vision-and-Language Tasks via Text Generation" (ICML 2021)

data_management_LLM

Collection of training data management explorations for large language models

A Chinese Open-Domain Dialogue System

LLMInternSkill: LLM internship resume and job-search Codex Skill for resume polish, JD tailoring, evidence…

🔗 Famiglie affini

instruction-tuning 28 language-model 113 dialogue 10 pre-training 11 chinese 70 attention-mechanism 54 from-scratch 7 tokenizer 5

Misurato dai temi di GitHub condivisi da entrambi i progetti, ponderato in base a quanto è raro ciascun tema.