A Full Commentary Of The Transformer Paper

I started a full commentary of the Transformer paper. I made a bet to myself: this should be the best resource on the whole internet to understand the paper in full, even to complete beginners.

It currently already covers self-attention with worked example and code. It covers each step only as much as we need to understand the paper.

It’s a great opportunity to write beautiful maths and draw awesome ASCII art.

I remember trying everything to learn about transformers:

• loads of ml courses
• 3Blue1Brown vids
• loads of tutorials
• the annotated transformer
• …

But, it either did not click or i did not remember clearly enough to explain. The best way according to me is to read the original paper which is clear once you understand it.

The Annotated Transformer is a annotated version of the paper with the full working code. But, this does not help with understanding the paper for people not familiar with the field.

Check it out: https://www.compileralchemy.com/annotated-transformer-commentary/

Request to read and give feedback. This is an OpenSource commentary. Feel free to contribute.