LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
Sebastian Raschka's companion repository to "Build a Large Language Model (From Scratch)": you implement attention, a transformer, pretraining and fine-tuning yourself, in plain PyTorch.
From the README
git clone --depth 1 https://github.com/rasbt/LLMs-from-scratch.git
- Link to the official source code repository
- Link to the book at Manning (the publisher's website)
- Link to the book page on Amazon.com
Excerpts from the project README on GitHub. Copyright and licensing remain with the respective authors.
Claim its page: indexed whatever its rank, translated into six languages, and enriched with what you write yourself.
Get an email alert on its next release or when it starts trending — never miss the moment.
Free · no card · unsubscribe anytimeImplement a ChatGPT-like LLM in PyTorch from scratch, step by step
LLMs-from-scratch has 104k stars on GitHub. It has been forked 15.9k times. LLMs-from-scratch is written mainly in Jupyter Notebook. It has been in active development since 2023. Its main topics are ai, artificial-intelligence, attention-mechanism, deep-learning.
Read the full guideWhat is LLMs-from-scratch?
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
Is LLMs-from-scratch open source?
LLMs-from-scratch is an open-source project.
Is LLMs-from-scratch free?
Yes. LLMs-from-scratch is free and open source — you can use, modify and self-host it.
What language is LLMs-from-scratch written in?
LLMs-from-scratch is written mainly in Jupyter Notebook.
Add this live badge to your README — your GitHub stars and directory rank, refreshed daily.
[](https://olud.ai/project/rasbt-llms-from-scratch.html)
Measured from GitHub topics shared by both projects, weighted by how rare each topic is.