pretraining

13 projetos partilham este topic do GitHub

Linhas conectam membros que estão mensuravelmente relacionados entre si. O tamanho do ponto reflete estrelas.

🧬 Membros

LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

General technology for enabling AI capabilities w/ LLMs and MLLMs

mPLUG-Owl: The Powerful Multi-modal Large Language Model Family

HRM-Text is a 1B text generation model based on the HRM architecture, strengthened by task completion and…

The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models

Official repository for "Craw4LLM: Efficient Web Crawling for LLM Pretraining"

[NeurIPS 2025 Spotlight] TPA: Tensor ProducT ATTenTion Transformer (https://arxiv.org/abs/2501.06425)

BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-training

PyTorch code for "Unifying Vision-and-Language Tasks via Text Generation" (ICML 2021)

data_management_LLM

Collection of training data management explorations for large language models

A Chinese Open-Domain Dialogue System

LLMInternSkill: LLM internship resume and job-search Codex Skill for resume polish, JD tailoring, evidence…

🔗 Familias relacionadas

instruction-tuning 28 language-model 113 dialogue 10 pre-training 11 chinese 70 attention-mechanism 54 from-scratch 7 tokenizer 5

Medido a partir dos tópicos do GitHub compartilhados por ambos os projetos, ponderado pela raridade de cada tópico.