A deep understanding of AI large language model mechanisms
An intensive masterclass on the internal mechanisms of Transformers and Large Language Models, including attention, tokenization, and scaling laws. Teaches how to build LLM components from scratch and fine-tune models for custom NLP tasks.
Instructor
Mike X Cohen
Duration
91 hours
The certificate
In progress — no credential yet
What it covered — 11 modules
- 01
Introduction
Welcome to the course! · What are LLMs? · Course philosophy and how to succeed
4/4 lectures - 02
Tokens and Tokenization
Introduction to tokens · Character-level vs. Word-level tokenization · Subword tokenization: BPE, WordPiece, and Unigram
7/7 lectures - 03
Embeddings
The geometry of meaning: What are embeddings? · One-hot encoding vs. Dense embeddings · Word2Vec: CBOW and Skip-gram
7/7 lectures - 04
The Transformer Architecture: Self-Attention
The 'Attention is All You Need' revolution · The intuition of Self-Attention · Dot-product attention: The Math
0/7 lectures - 05
Multi-Head Attention (MHA)
Why one head isn't enough · The architecture of Multi-Head Attention · Linear projections and Concatenation
0/5 lectures - 06
Positional Encoding
Why Transformers need position information · Sinusoidal positional encoding · Learned positional embeddings
0/5 lectures - 07
The Feed-Forward Network and Normalization
The Position-wise Feed-Forward Network (FFN) · Layer Normalization vs. Batch Normalization · RMSNorm (Root Mean Square Normalization)
0/4 lectures - 08
Decoder-Only Models (GPT-style)
Encoder vs. Decoder architectures · Masked Self-Attention · Causal modeling
0/4 lectures - 09
Training LLMs
Pre-training: Next Token Prediction · Loss functions: Cross-Entropy in LLMs · Weight initialization strategies
0/5 lectures - 10
Evaluation and Metrics
Perplexity: The standard for LLMs · BLEU and ROUGE scores · Human evaluation and Benchmarks (MMLU, GSM8K)
0/4 lectures - 11
Advanced Topics and Interpretability
Attention Map visualization · Saliency and Logit Lens · Intervention studies
0/4 lectures
Toolkit from this course
More certificates