ML Refresher

Contents

ML Refresher

Mathematics, statistics & deep learning

Single-page editionDownload PDF

Contents

44 chapters · 6 appendices

Part IMathematics for Machine Learning

  1. 01Mathematical Foundations
  2. 02Trigonometry
  3. 03Calculus
  4. 04Linear Algebra
  5. 05Vector Calculus
  6. 06Probability Theory
  7. 07Information Theory
  8. 08Hypothesis Testing

Part IINeural Network Fundamentals

  1. 09Learning from Data
  2. 10Automatic Differentiation
  3. 11Activation Functions
  4. 12Softmax & Cross-Entropy
  5. 13Loss Functions & Divergences
  6. 14Neural Networks from Scratch
  7. 15Optimizers & Schedules
  8. 16Normalization, Residuals & Precision

Part IIITransformers from Scratch

  1. 17Tokenization & Embeddings
  2. 18Language Modeling
  3. 19Scaled Dot-Product Attention
  4. 20Multi-Head Attention
  5. 21Positional Encoding & RoPE
  6. 22The Transformer Block
  7. 23Training a GPT from Scratch

Part IVModern LLM Architecture

  1. 24KV Cache & Grouped-Query Attention
  2. 25Multi-Head Latent Attention
  3. 26Online Softmax & FlashAttention
  4. 27Mixture of Experts
  5. 28Linear Attention & State-Space Models
  6. 29Scaling Laws & Pretraining Recipes

Part VRepresentation & Multimodal Learning

  1. 30Contrastive & Metric Learning
  2. 31Vision Transformers
  3. 32Vision-Language Models

Part VIPost-Training & Reinforcement Learning

  1. 33Supervised Fine-Tuning & LoRA
  2. 34Reinforcement Learning Foundations
  3. 35Reward Models, PPO & RLHF
  4. 36Direct Preference Optimization
  5. 37GRPO & Verifiable Rewards
  6. 38Distillation & Reasoning Models

Part VIIInference & Systems

  1. 39Decoding & Speculative Sampling
  2. 40Quantization & Serving
  3. 41Training at Scale

Part VIIIAgents

  1. 42Tool Use & Agent Loops
  2. 43Retrieval, Memory, Planning & Evaluation
  3. 44Capstone: An LLM End to End

Appendices

  1. ANotation & Shapes
  2. BNumPy for Deep Learning
  3. CMatrix Calculus Cookbook
  4. DSolutions to Exercises
  5. EFormula Sheets
  6. FBibliography