Home› Projects› pytorch-original-transformer
pytorch-original-transformer
Jupyter Notebook

pytorch-original-transformer

My implementation of the original transformer model (Vaswani et al.). I've additionally included the playground.py file for visualizing otherwise seemingly hard concepts. Currently included IWSLT pretrained models.

by gordicaleksa · GitHub
Stars
Forks
License
Created
Last commit
Category
attentionattention-is-all-you-needattention-mechanismMITJupyter Notebook
View on GitHub
In plain words

Experiment with a basic version of the transformer model to learn how it processes language.

You maintain this project?

Claim its page: indexed whatever its rank, translated into six languages, and enriched with what you write yourself.

Claim this page →
pytorch-original-transformer — GitHub preview card
📈 Star history
1 1101 109
2026-07-202026-08-31
📈 Track pytorch-original-transformer

Get an email alert on its next release or when it starts trending — never miss the moment.

Free · no card · unsubscribe anytime
Get email alerts →
📄 About

My implementation of the original transformer model (Vaswani et al.). I've additionally included the playground.py file for visualizing otherwise seemingly hard concepts. Currently included IWSLT pretrained models.

pytorch-original-transformer has 1.1k stars on GitHub. It has been forked 187 times. pytorch-original-transformer is written mainly in Jupyter Notebook. It has been in active development since 2020. pytorch-original-transformer is available under the MIT license. Its main topics are attention, attention-is-all-you-need, attention-mechanism, deep-learning.

❓ Frequently asked questions

What is pytorch-original-transformer?

My implementation of the original transformer model (Vaswani et al.). I've additionally included the playground.py file for visualizing otherwise seemingly hard concepts. Currently included IWSLT pretrained models.

Is pytorch-original-transformer open source?

pytorch-original-transformer is an open-source project. It is released under the MIT license.

Is pytorch-original-transformer free?

Yes. pytorch-original-transformer is free and open source — you can use, modify and self-host it.

What license does pytorch-original-transformer use?

pytorch-original-transformer is available under the MIT license.

What language is pytorch-original-transformer written in?

pytorch-original-transformer is written mainly in Jupyter Notebook.

🏅 Maintainer of this project?
olud.ai badge — pytorch-original-transformer

Add this live badge to your README — your GitHub stars and directory rank, refreshed daily.

[![olud.ai](https://olud.ai/badge.php?tool=gordicaleksa-pytorch-original-transformer)](https://olud.ai/project/gordicaleksa-pytorch-original-transformer.html)
More badge options →
🧬 Shares DNA with🧬 View the DNA map →
attention-is-all-you-need-pytorch
A PyTorch implementation of the Transformer model in "Attention is All You Need".
★ 9.8k · attention
sharesattention-is-all-you-needattention
self-attention-cv
Implementation of various self-attention mechanisms focused on computer vision. Ongoing reposit…
★ 1.2k · artificial-intelligence
sharesattentionattention-mechanism
performer-pytorch
An implementation of Performer, a linear attention-based transformer, in Pytorch
★ 1.2k · artificial-intelligence
sharesattentionattention-mechanism
lambda-networks
Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with le…
★ 1.5k · artificial-intelligence
sharesattentionattention-mechanism
PALM-E
Implementation of "PaLM-E: An Embodied Multimodal Language Model"
★ 338 · artificial-intelligence
sharesattention-is-all-you-needattention-mechanism
whisper-timestamped
Multilingual Automatic Speech Recognition with word-level timestamps and confidence
★ 2.8k · asr
sharesattention-is-all-you-needattention-mechanism
Deepdive-llama3-from-scratch
Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivati…
★ 631 · attention
sharesattentionattention-mechanism
nanodl
JAX library for training sub-4B foundation models for edge
★ 306 · attention
sharesattentionattention-mechanism
CM3Leon
An open source implementation of "Scaling Autoregressive Multi-Modal Models: Pretraining and In…
★ 365 · attention
sharesattention-is-all-you-needattention
pytorch-GAT
My implementation of the original GAT paper (Veličković et al.). I've additionally included the…
★ 2.7k · attention
sharesattentionattention-mechanism

Measured from GitHub topics shared by both projects, weighted by how rare each topic is.