ML Refresher

Appendix E

Formula Sheets

The key equations from each chapter, collected for review.

6 Probability Theory

Key equations
P(a<X<b)=∫abp(x) dx,p(y∣x)=p(x,y)p(x),p(h∣e)∝p(e∣h) p(h)P(a < X < b) = \int_a^b p(x)\,\dd x, \qquad p(y \mid x) = \frac{p(x, y)}{p(x)}, \qquad p(h \mid e) \propto p(e \mid h)\, p(h)
p(x1,…,xT)=∏tp(xt∣x<t)p(x_1, \dots, x_T) = \prod_t p(x_t \mid x_{<t})
E[aX+bY]=aE[X]+bE[Y],Var⁡[X]=E[X2]−E[X]2,Var⁡[Xˉn]=σ2/n\E[aX + bY] = a\E[X] + b\E[Y], \qquad \Var[X] = \E[X^2] - \E[X]^2, \qquad \Var[\bar{X}_n] = \sigma^2 / n
∇θEpθ[f(x)]=Epθ[f(x) ∇θlog⁡pθ(x)]\nabla_\theta \E_{p_\theta}[f(x)] = \E_{p_\theta}[f(x)\, \nabla_\theta \log p_\theta(x)]
θ^=arg min⁡θ−1N∑ilog⁡pθ(yi∣xi);Gaussian→MSE,  Categorical→cross-entropy\hat{\vtheta} = \argmin_\vtheta -\tfrac1N \textstyle\sum_i \log p_\vtheta(y_i \mid x_i); \quad \text{Gaussian} \to \text{MSE}, \; \text{Categorical} \to \text{cross-entropy}
arg max⁡k(zk+gk)∼softmax⁡(z),gk=−log⁡(−log⁡uk)\argmax_k (z_k + g_k) \sim \softmax(\vz), \qquad g_k = -\log(-\log u_k)

Read Chapter 6

7 Information Theory

Key equations
I(x)=−log⁡p(x),H(p)=−∑xp(x)log⁡p(x)≤log⁡KI(x) = -\log p(x), \qquad H(p) = -\sum_x p(x)\log p(x) \le \log K
H(p,q)=−∑xp(x)log⁡q(x)=H(p)+DKL(p∥q),DKL(p∥q)=∑xp(x)log⁡p(x)q(x)≥0H(p, q) = -\sum_x p(x) \log q(x) = H(p) + \KL(p \Vert q), \qquad \KL(p \Vert q) = \sum_x p(x)\log\frac{p(x)}{q(x)} \ge 0
−1N∑ilog⁡qθ(xi)=H(p^,qθ),PPL⁡=exp⁡(cross-entropy per token)-\frac1N\sum_i \log q_\vtheta(x_i) = H(\hat{p}, q_\vtheta), \qquad \operatorname{PPL} = \exp(\text{cross-entropy per token})
I(X;Y)=DKL(p(x,y)∥p(x)p(y))=H(X)−H(X∣Y)I(X; Y) = \KL(p(x, y) \Vert p(x)p(y)) = H(X) - H(X \mid Y)
DKL(q∥p)=Ex∼q[(r−1)−log⁡r],r=p(x)/q(x)\KL(q \Vert p) = \E_{x \sim q}\big[(r - 1) - \log r\big], \quad r = p(x)/q(x)

Read Chapter 7

8 Hypothesis Testing

Key equations
p^=1N∑iYi\hat p = \frac1N\sum_i Y_i
SE^(p^)=p^(1−p^)N\widehat{\mathrm{SE}}(\hat p) = \sqrt{\frac{\hat p(1-\hat p)}{N}}
normal CI=p^±1.96 SE^(p^)\text{normal CI} = \hat p \pm 1.96\,\widehat{\mathrm{SE}}(\hat p)
p-value=PH0(∣T∣≥∣Tobs∣)p\text{-value} = P_{H_0}(|T| \ge |T_{\mathrm{obs}}|)
N≈1.962p(1−p)h2N \approx \frac{1.96^2p(1-p)}{h^2}

Read Chapter 8

9 Learning from Data

Key equations
R^(θ)=1N∑iℓ(fθ(xi),yi)\hat R(\vtheta) = \frac1N\sum_i \ell(f_\vtheta(\vx_i), y_i)
∇θ1N∥Xθ−y∥2=2NX⊤(Xθ−y)\nabla_\vtheta \frac1N\|\mX\vtheta-\vy\|^2 = \frac2N\mX^\T(\mX\vtheta-\vy)
X⊤Xθ=X⊤y\mX^\T\mX\vtheta = \mX^\T\vy
θ←θ−η∇θL\vtheta \leftarrow \vtheta - \eta\nabla_\vtheta L
∇wLBCE=1NX⊤(p−y)\nabla_\vw L_{\mathrm{BCE}} = \frac1N\mX^\T(\vp-\vy)

Read Chapter 9

10 Automatic Differentiation

Key equations
v˙=Jf(u)u˙\dot{\vv} = \mJ_f(\vu)\dot{\vu}
uˉ=vˉ Jf(u)\bar{\vu} = \bar{\vv}\,\mJ_f(\vu)
z=x+y:xˉ+=zˉ,  yˉ+=zˉz=x+y: \quad \bar{x} \mathrel{+}= \bar{z},\; \bar{y} \mathrel{+}= \bar{z}
z=xy:xˉ+=zˉy,  yˉ+=zˉxz=xy: \quad \bar{x} \mathrel{+}= \bar{z}y,\; \bar{y} \mathrel{+}= \bar{z}x
Y=AW:Aˉ=YˉW⊤,  Wˉ=A⊤Yˉ\mY=\mA\mW: \quad \bar{\mA}=\bar{\mY}\mW^\T,\; \bar{\mW}=\mA^\T\bar{\mY}

Read Chapter 10

11 Activation Functions

Key equations
σ′(x)=σ(x)(1−σ(x)),tanh⁡′(x)=1−tanh⁡2(x)\sigma'(x)=\sigma(x)(1-\sigma(x)), \qquad \tanh'(x)=1-\tanh^2(x)
ReLU⁡′(x)=1{x>0},softplus⁡′(x)=σ(x)\operatorname{ReLU}'(x)=\mathbf{1}\{x>0\}, \qquad \operatorname{softplus}'(x)=\sigma(x)
GELU⁡(x)=xΦ(x),GELU⁡′(x)=Φ(x)+xϕ(x)\operatorname{GELU}(x)=x\Phi(x), \qquad \operatorname{GELU}'(x)=\Phi(x)+x\phi(x)
SiLU⁡(x)=xσ(x),SiLU⁡′(x)=σ(x)+xσ(x)(1−σ(x))\operatorname{SiLU}(x)=x\sigma(x), \qquad \operatorname{SiLU}'(x)=\sigma(x)+x\sigma(x)(1-\sigma(x))
FFN⁡(x)=(a(xW)⊙xV)W2,h=8d/3\operatorname{FFN}(\vx)=(a(\vx\mW)\odot\vx\mV)\mW_2, \qquad h=8d/3

Read Chapter 11

12 Softmax & Cross-Entropy

Key equations
pi=ezi/T∑jezj/T,softmax⁡(z+c1)=softmax⁡(z)p_i=\frac{e^{z_i/T}}{\sum_j e^{z_j/T}}, \qquad \softmax(\vz+c\one)=\softmax(\vz)
log⁡pi=zi−m−log⁡∑jezj−m,m=max⁡jzj\log p_i=z_i-m-\log\sum_j e^{z_j-m}, \qquad m=\max_j z_j
Jsoftmax⁡=diag⁡(p)−pp⊤J_{\softmax}=\diag(\vp)-\vp\vp^\T
L=−∑iyilog⁡pi,∇zL=p−yL=-\sum_i y_i\log p_i, \qquad \nabla_{\vz}L=\vp-\vy
yϵ=(1−ϵ)y+ϵ1/K,∇zλ(log⁡Z)2=2λlog⁡Z p\vy^\epsilon=(1-\epsilon)\vy+\epsilon\one/K, \qquad \nabla_{\vz}\lambda(\log Z)^2=2\lambda\log Z\,\vp

Read Chapter 12

13 Loss Functions & Divergences

Key equations
N(y^,σ2)⇒r2,Laplace⁡(y^,b)⇒∣r∣\mathcal{N}(\hat y,\sigma^2)\Rightarrow r^2,\qquad \operatorname{Laplace}(\hat y,b)\Rightarrow |r|
ℓδ(r)={12r2,∣r∣≤δδ(∣r∣−12δ),∣r∣>δ\ell_\delta(r)= \begin{cases}\tfrac12r^2,& |r|\le\delta\\ \delta(|r|-\tfrac12\delta),& |r|>\delta \end{cases}
LBCE=max⁡(z,0)−zy+log⁡(1+e−∣z∣),∇zL=σ(z)−yL_{\mathrm{BCE}}=\max(z,0)-zy+\log(1+e^{-|z|}),\qquad \nabla_z L=\sigma(z)-y
Lfocal=−αt(1−pt)γlog⁡ptL_{\mathrm{focal}}=-\alpha_t(1-p_t)^\gamma\log p_t
∇zDKL(p∥qθ)=qθ−p,∇zLKD=T(qT−pT)\nabla_{\vz}\KL(p\Vert q_\theta)=q_\theta-p,\qquad \nabla_{\vz}L_{\mathrm{KD}}=T(q_T-p_T)

Read Chapter 13

14 Neural Networks from Scratch

Key equations
Z1=XW1+b1,H=max⁡(0,Z1)\mZ_1 = \mX\mW_1 + \vb_1,\quad \mH = \max(0, \mZ_1)
L=−1B∑ilog⁡Pi,yiL = -\frac{1}{B}\sum_i \log P_{i,y_i}
Zˉ2=(P−Y)/B\bar{\mZ}_2 = (\mP - \mY)/B
Wˉ=A⊤Zˉ,Aˉ=ZˉW⊤\bar{\mW} = \mA^\T\bar{\mZ},\quad \bar{\mA}=\bar{\mZ}\mW^\T
Var⁡ ⁣[∑iwixi]=n Var⁡[w]Var⁡[x]\Var\!\left[\sum_i w_i x_i\right] = n\,\Var[w]\Var[x]

Read Chapter 14

15 Optimizers & Schedules

Key equations
θt+1=θt−ηgt\vtheta_{t+1}=\vtheta_t-\eta g_t
vt=βvt−1−ηgt\vv_t=\beta\vv_{t-1}-\eta g_t
m^t=mt/(1−β1t),v^t=vt/(1−β2t)\hat{m}_t=m_t/(1-\beta_1^t),\quad \hat{v}_t=v_t/(1-\beta_2^t)
θ←(1−ηλ)θ−η m^/(v^+ϵ)\vtheta \leftarrow (1-\eta\lambda)\vtheta -\eta\,\hat{m}/(\sqrt{\hat{v}}+\epsilon)
g←gmin⁡(1,c/(∥g∥2+ϵ))g \leftarrow g\min(1,c/(\|g\|_2+\epsilon))

Read Chapter 15

16 Normalization, Residuals & Precision

Key equations
x^=(x−μ)/σ2+ϵ\hat{\vx}=(\vx-\mu)/\sqrt{\sigma^2+\epsilon}
RMSNorm⁡(x)=γ⊙x/1d∑jxj2+ϵ\operatorname{RMSNorm}(\vx)=\boldsymbol{\gamma}\odot \vx / \sqrt{\frac1d\sum_j x_j^2+\epsilon}
xl+1=xl+Fl(xl)\vx_{l+1}=\vx_l+F_l(\vx_l)
Dropout⁡(x)=m⊙x/pkeep\operatorname{Dropout}(\vx)=\vm\odot\vx/p_{\mathrm{keep}}
gtrue=(Sg)/Sg_{\mathrm{true}}=(Sg)/S

Read Chapter 16

17 Tokenization & Embeddings

Key equations
UTF-8 text→(b1,…,bn),bi∈{0,…,255}\text{UTF-8 text} \rightarrow (b_1,\ldots,b_n), \qquad b_i \in \{0,\ldots,255\}
(a,b)=arg max⁡(u,v)count⁡(u,v)(a,b) = \argmax_{(u,v)} \operatorname{count}(u,v)
xi=eidiE=Eidi,:\vx_i = \boldsymbol{e}_{\text{id}_i}\mE = \mE_{\text{id}_i,:}
Eˉj=∑i: idi=jxˉi\bar{\mE}_j = \sum_{i:\,\text{id}_i=j}\bar{\vx}_i
zt=htE⊤(tied output weights)\vz_t = \vh_t\mE^\T \quad \text{(tied output weights)}

Read Chapter 17

18 Language Modeling

Key equations
p(x1,…,xT)=∏t=1Tp(xt∣x<t)p(x_1,\ldots,x_T)=\prod_{t=1}^{T}p(x_t\mid x_{<t})
q(j∣i)=cij+α∑kcik+αVq(j\mid i)=\frac{c_{ij}+\alpha}{\sum_k c_{ik}+\alpha V}
L=−1N∑nlog⁡qn,yn,zˉn=qn−eynN\mathcal{L}=-\frac1N\sum_n\log q_{n,y_n}, \qquad \bar{\vz}_n=\frac{\vq_n-\boldsymbol{e}_{y_n}}{N}
PPL⁡=exp⁡(1N∑n−log⁡qn,yn)\operatorname{PPL}=\exp\left(\frac{1}{N}\sum_n-\log q_{n,y_n}\right)
qt=softmax⁡(tanh⁡([Ext−C;…;Ext−1]W1+b1)W2+b2)\vq_t=\softmax(\tanh([\mE_{x_{t-C}};\ldots;\mE_{x_{t-1}}]\mW_1+\vb_1)\mW_2+\vb_2)

Read Chapter 18

19 Scaled Dot-Product Attention

Key equations
S=QK⊤/dk,A=softmax⁡(S),O=AV\mS=\mQ\mK^\T/\sqrt{d_k},\qquad \mA=\softmax(\mS),\qquad \mO=\mA\mV
Var⁡(∑ℓ=1dkqℓkℓ)=dk\Var\left(\sum_{\ell=1}^{d_k}q_\ell k_\ell\right)=d_k
Vˉ=A⊤Oˉ,Aˉ=OˉV⊤\bar{\mV}=\mA^\T\bar{\mO},\qquad \bar{\mA}=\bar{\mO}\mV^\T
Sˉ=A⊙(Aˉ−rowsum⁡(Aˉ⊙A))\bar{\mS}=\mA\odot\left(\bar{\mA} -\operatorname{rowsum}(\bar{\mA}\odot\mA)\right)
Qˉ=SˉK/dk,Kˉ=Sˉ⊤Q/dk\bar{\mQ}=\bar{\mS}\mK/\sqrt{d_k},\qquad \bar{\mK}=\bar{\mS}^\T\mQ/\sqrt{d_k}

Read Chapter 19

20 Multi-Head Attention

Key equations
Q=XqWQ,K=XkWK,V=XvWV\mQ=\mX_q\mW_Q,\qquad \mK=\mX_k\mW_K,\qquad \mV=\mX_v\mW_V
Oh=softmax⁡(QhKh⊤/dh)Vh\mO_h=\softmax(\mQ_h\mK_h^\T/\sqrt{d_h})\mV_h
Y=concat⁡(O1,…,OH)WO\mY=\operatorname{concat}(\mO_1,\ldots,\mO_H)\mW_O
#parameters=4d2,FLOPs≈4BTd2+2BT2d\#\text{parameters}=4d^2,\qquad \text{FLOPs}\approx 4BTd^2+2BT^2d
Xˉself=Xˉq+Xˉk+Xˉv\bar{\mX}_{\text{self}}=\bar{\mX}_q+\bar{\mX}_k+\bar{\mX}_v

Read Chapter 20

21 Positional Encoding & RoPE

Key equations
Attn⁡(Q,K,V)=softmax⁡(QK⊤/dh)V\operatorname{Attn}(\mQ,\mK,\mV) = \softmax(\mQ\mK^{\T}/\sqrt{d_h})\mV
pt,2i=sin⁡(tθi),pt,2i+1=cos⁡(tθi)p_{t,2i}=\sin(t\theta_i), \quad p_{t,2i+1}=\cos(t\theta_i)
θi=b−2i/d\theta_i=b^{-2i/d}
⟨Rmq,Rnk⟩=q⊤Rn−mk\langle R_m\vq, R_n\vk\rangle = \vq^{\T}R_{n-m}\vk
ALiBi⁡h,t,s=−αh(t−s)(s≤t)\operatorname{ALiBi}_{h,t,s}=-\alpha_h(t-s) \quad (s\le t)

Read Chapter 21

22 The Transformer Block

Key equations
uℓ=xℓ+Attn⁡(RMSNorm⁡(xℓ))\vu_\ell = \vx_\ell + \operatorname{Attn}(\operatorname{RMSNorm}(\vx_\ell))
xℓ+1=uℓ+FFN⁡(RMSNorm⁡(uℓ))\vx_{\ell+1}=\vu_\ell+\operatorname{FFN}(\operatorname{RMSNorm}(\vu_\ell))
FFN⁡(x)=(SiLU⁡(xWg)⊙xWu)Wd\operatorname{FFN}(\vx)=(\operatorname{SiLU}(\vx\mW_g)\odot\vx\mW_u)\mW_d
Pblock≈4d2+3dhP_{\text{block}} \approx 4d^2 + 3dh
Ctrain≈6NDC_{\text{train}} \approx 6ND

Read Chapter 22

23 Training a GPT from Scratch

Key equations
p(x1,…,xT)=∏tp(xt∣x<t)p(x_1,\ldots,x_T)=\prod_t p(x_t\mid x_{<t})
Zb,t=hb,tE⊤\mZ_{b,t}=\vh_{b,t}\mE^{\T}
L=−1BT∑b,tlog⁡softmax⁡(Zb,t)yb,t\mathcal{L}=-\frac{1}{BT}\sum_{b,t}\log\softmax(\mZ_{b,t})_{y_{b,t}}
ηs=ηmax⁡min⁡(1,s/Swarmup)\eta_s=\eta_{\max}\min(1, s/S_{\text{warmup}})
pi=softmax⁡(zi/τ)after optional top-k maskingp_i=\softmax(z_i/\tau) \quad \text{after optional top-}k\text{ masking}

Read Chapter 23

24 KV Cache & Grouped-Query Attention

Key equations
at=∑s≤tsoftmax⁡s ⁣(qt⊤ks/dh)vs\va_t = \sum_{s \le t}\softmax_s\!\left(\vq_t^\T\vk_s/\sqrt{d_h}\right)\vv_s
MKV=2 L G dh T bM_{KV} = 2\,L\,G\,d_h\,T\,b
g(h)=⌊hG/H⌋,1≤G≤Hg(h) = \lfloor hG/H \rfloor, \qquad 1 \le G \le H
visible(t,s)=(s≤t)∧(s≥t−W+1  ∨  s<S)\text{visible}(t, s) = (s \le t) \land (s \ge t-W+1 \;\lor\; s < S)

Read Chapter 24

25 Multi-Head Latent Attention

Key equations
cs=xsWDKV\vc_s = \vx_s \mW_{DKV}
ks,h=csWUK,h,vs,h=csWUV,h\vk_{s,h} = \vc_s\mW_{UK,h}, \qquad \vv_{s,h} = \vc_s\mW_{UV,h}
qt⊤ks=(qtWUK⊤)⋅cs\vq_t^\T\vk_s = (\vq_t\mW_{UK}^\T)\cdot\vc_s
MMLA=L T dc bM_{MLA} = L\,T\,d_c\,b

Read Chapter 25

26 Online Softmax & FlashAttention

Key equations
m=max⁡isi,ℓ=∑iesi−mm = \max_i s_i, \qquad \ell = \sum_i e^{s_i-m}
ℓnew=emold−mnewℓold+emB−mnewℓB\ell_{new} = e^{m_{old}-m_{new}}\ell_{old} + e^{m_B-m_{new}}\ell_B
nnew=emold−mnewnold+emB−mnew∑j∈Besj−mBvj\vn_{new} = e^{m_{old}-m_{new}}\vn_{old} + e^{m_B-m_{new}}\sum_{j\in B} e^{s_j-m_B}\vv_j
attn⁡(q,K,V)=n/ℓ\operatorname{attn}(\vq, \mK, \mV) = \vn / \ell

Read Chapter 26

27 Mixture of Experts

Key equations
p=softmax⁡(r),S=topk⁡(p)\vp = \softmax(\vr), \qquad S = \operatorname{topk}(\vp)
ai=pi∑j∈Spj,y=∑i∈SaiEi(x)a_i = \frac{p_i}{\sum_{j\in S}p_j}, \quad \vy = \sum_{i\in S} a_i E_i(\vx)
Llb=N∑ifiPi,N∑ipi2≥1\mathcal{L}_{\text{lb}} = N\sum_i f_iP_i, \qquad N\sum_i p_i^2 \ge 1
Lz=E[(log⁡∑ieri)2]\mathcal{L}_z = \E\Big[\big(\log\sum_i e^{r_i}\big)^2\Big]
bi←bi−η sign⁡(fi−1/N)b_i \leftarrow b_i - \eta\,\sign(f_i - 1/N)

Read Chapter 27

28 Linear Attention & State-Space Models

Key equations
K(q,k)=ϕ(q)⊤ϕ(k)K(\vq,\vk) = \vphi(\vq)^\T\vphi(\vk)
St=St−1+ϕ(kt)vt⊤,ct=ct−1+ϕ(kt)\mS_t = \mS_{t-1} + \vphi(\vk_t)\vv_t^\T, \quad \vc_t = \vc_{t-1} + \vphi(\vk_t)
yt=ϕ(qt)⊤Stϕ(qt)⊤ct\vy_t = \frac{\vphi(\vq_t)^\T\mS_t} {\vphi(\vq_t)^\T\vc_t}
St=St−1+βtϕ(kt)(vt−ϕ(kt)⊤St−1)⊤\mS_t = \mS_{t-1} + \beta_t\vphi(\vk_t) (\vv_t - \vphi(\vk_t)^\T\mS_{t-1})^\T
ht=Aˉtht−1+Bˉtxt,yt=Ctht\vh_t = \bar{\mA}_t\vh_{t-1} + \bar{\mB}_t\vx_t, \quad \vy_t = \mC_t\vh_t

Read Chapter 28

29 Scaling Laws & Pretraining Recipes

Key equations
C≈2ND+4ND=6NDC \approx 2ND + 4ND = 6ND
log⁡y=log⁡a−αlog⁡x\log y = \log a - \alpha\log x
L(N,D)=E+A/Nα+B/DβL(N,D)=E + A/N^\alpha + B/D^\beta
αAN−α=βBD−β,ND=C/6\alpha A N^{-\alpha} = \beta B D^{-\beta}, \qquad ND=C/6
D≈20N(planning rule of thumb)D \approx 20N \quad \text{(planning rule of thumb)}

Read Chapter 29

30 Contrastive & Metric Learning

Key equations
s(z,w)=z⊤w∥z∥ ∥w∥,d=1−ss(\vz, \vw) = \frac{\vz^\T\vw}{\|\vz\|\,\|\vw\|}, \qquad d = 1 - s
Lpair=yd2+(1−y)max⁡(0,m−d)2L_{\mathrm{pair}} = y d^2 + (1-y)\max(0, m-d)^2
Ltriplet=max⁡(0,d(a,p)−d(a,n)+m)L_{\mathrm{triplet}} = \max(0, d(a,p)-d(a,n)+m)
Li=−log⁡softmax⁡(Si/τ)i,Sˉij=Pij−1[i=j]NτL_i = -\log\softmax(S_i/\tau)_i, \qquad \bar{S}_{ij} = \frac{P_{ij}-\one[i=j]}{N\tau}
LSigLIP=N−2∑i,jlog⁡(1+e−Yij(Sij+b))L_{\mathrm{SigLIP}} = N^{-2}\sum_{i,j}\log(1+e^{-Y_{ij}(S_{ij}+b)})

Read Chapter 30

31 Vision Transformers

Key equations
GH=H/P,GW=W/P,T=GHGWG_H = H/P, \qquad G_W = W/P, \qquad T = G_HG_W
Xpatch∈RB×T×P2C\mX_{\mathrm{patch}} \in \R^{B \times T \times P^2C}
zt=Xpatch,tWE+bE+pt\vz_t = \mX_{\mathrm{patch},t}\mW_E + \vb_E + \vp_t
Attention⁡(Q,K,V)=softmax⁡(QK⊤/dh)V\operatorname{Attention}(\mQ,\mK,\mV) = \softmax(\mQ\mK^\T/\sqrt{d_h})\mV
himage=hclsorhimage=T−1∑tht\vh_{\mathrm{image}} = \vh_{\mathrm{cls}} \quad\text{or}\quad \vh_{\mathrm{image}} = T^{-1}\sum_t \vh_t

Read Chapter 31

32 Vision-Language Models

Key equations
p(c∣i)=softmax⁡c(α i⊤tc)p(c \mid \vi) = \softmax_c(\alpha\, \vi^\T\vt_c)
Himg=ZvisionWP+bP\mH_{\mathrm{img}} = \mZ_{\mathrm{vision}}\mW_P + \vb_P
Qlearnedattends to⁡Zvision→Q tokens\mQ_{\mathrm{learned}} \operatorname{ attends\ to } \mZ_{\mathrm{vision}} \rightarrow Q \text{ tokens}
X′=X+tanh⁡(g)CrossAttn⁡(X,V),g=0⇒X′=X\mX' = \mX + \tanh(g)\operatorname{CrossAttn}(\mX,\mV), \qquad g=0 \Rightarrow \mX'=\mX
(F,H,W)↦(t,h,w),2 by 2 merge: HW↦HW/4(F,H,W) \mapsto (t,h,w), \qquad \text{2 by 2 merge: } HW \mapsto HW/4

Read Chapter 32

33 Supervised Fine-Tuning & LoRA

Key equations
L=−1M∑tmtlog⁡pθ(yt∣y<t),M=∑tmtL = -\frac{1}{M}\sum_t m_t \log p_\vtheta(y_t \mid y_{<t}), \qquad M=\sum_t m_t
zˉt,k=mtM(pt,k−1[k=yt])\bar{z}_{t,k}=\frac{m_t}{M}\big(p_{t,k}-\one[k=y_t]\big)
W′=W+αrBA,B0=0⇒W0′=W\mW' = \mW + \frac{\alpha}{r}\mB\mA, \qquad \mB_0=0 \Rightarrow \mW'_0=\mW
Bˉ=αrΔˉA⊤,Aˉ=αrB⊤Δˉ\bar{\mB}=\frac{\alpha}{r}\bar{\boldsymbol{\Delta}}\mA^\T, \qquad \bar{\mA}=\frac{\alpha}{r}\mB^\T\bar{\boldsymbol{\Delta}}
LoRA parameters=r(din+dout)≪dindout\text{LoRA parameters}=r(d_{in}+d_{out}) \ll d_{in}d_{out}

Read Chapter 33

34 Reinforcement Learning Foundations

Key equations
Gt=∑k=0∞γkrt+k,vπ(s)=Eπ[rt+γvπ(st+1)∣st=s]G_t=\sum_{k=0}^{\infty}\gamma^k r_{t+k}, \qquad v_\pi(s)=\E_\pi[r_t+\gamma v_\pi(s_{t+1})\mid s_t=s]
∇θJ=Eπ[∑tGt∇θlog⁡πθ(at∣st)]\nabla_\vtheta J = \E_\pi\Big[\sum_t G_t\nabla_\vtheta\log\pi_\vtheta(a_t\mid s_t)\Big]
Eπ[(Gt−b(st))∇log⁡π(at∣st)]=Eπ[Gt∇log⁡π(at∣st)]\E_\pi[(G_t-b(s_t))\nabla\log\pi(a_t\mid s_t)] = \E_\pi[G_t\nabla\log\pi(a_t\mid s_t)]
Eπ[f(a)]=Eb[π(a)b(a)f(a)]\E_\pi[f(a)] = \E_b\left[\frac{\pi(a)}{b(a)}f(a)\right]
A^tGAE=∑l=0∞(γλ)lδt+l,A^t=δt+γλA^t+1\hat{A}^{\mathrm{GAE}}_t=\sum_{l=0}^{\infty}(\gamma\lambda)^l\delta_{t+l}, \qquad \hat{A}_t=\delta_t+\gamma\lambda\hat{A}_{t+1}

Read Chapter 34

35 Reward Models, PPO & RLHF

Key equations
P(yw≻yl)=σ(rϕ(yw)−rϕ(yl)),ℓ(d)=−log⁡σ(d)P(y_w \succ y_l)=\sigma(r_\vphi(y_w)-r_\vphi(y_l)), \qquad \ell(d)=-\log\sigma(d)
J(θ)=Ey∼πθ[rϕ(x,y)]−βDKL(πθ(⋅∣x)∥πref(⋅∣x))J(\vtheta)=\E_{y\sim\pi_\vtheta}[r_\vphi(x,y)] -\beta\KL(\pi_\vtheta(\cdot\mid x)\Vert\pi_{ref}(\cdot\mid x))
rt(θ)=πθ(at∣st)πold(at∣st)r_t(\vtheta)=\frac{\pi_\vtheta(a_t\mid s_t)}{\pi_{old}(a_t\mid s_t)}
LtCLIP=min⁡(rtAt,clip⁡(rt,1−ϵ,1+ϵ)At)L^{CLIP}_t=\min\big(r_tA_t, \operatorname{clip}(r_t,1-\epsilon,1+\epsilon)A_t\big)
∇(rtAt)=Atrt∇log⁡πθ(at∣st)\nabla(r_tA_t)=A_t r_t\nabla\log\pi_\vtheta(a_t\mid s_t)

Read Chapter 35

36 Direct Preference Optimization

Key equations
J(π)=Eπ[r]−βDKL(π ∥ π0)J(\pi) = \E_\pi[r] - \beta\KL(\pi\,\Vert\,\pi_0)
π∗(y)=π0(y)er(y)/β/Z\pi^*(y) = \pi_0(y)e^{r(y)/\beta}/Z
r^θ(y)=βlog⁡πθ(y)π0(y)+βlog⁡Z\hat{r}_\theta(y) = \beta\log\frac{\pi_\theta(y)}{\pi_0(y)} + \beta\log Z
LDPO=−log⁡σ(r^w−r^l)\mathcal{L}_{\mathrm{DPO}} = -\log\sigma(\hat{r}_w - \hat{r}_l)
∇L=−βσ(r^l−r^w)∇(log⁡πw−log⁡πl)\nabla\mathcal{L} = -\beta\sigma(\hat{r}_l-\hat{r}_w) \nabla(\log\pi_w-\log\pi_l)

Read Chapter 36

37 GRPO & Verifiable Rewards

Key equations
Ai=(ri−rˉ)/(sr+ϵ)A_i = (r_i - \bar{r})/(s_r+\epsilon)
ρi=exp⁡(log⁡πθ(yi)−log⁡πold(yi))\rho_i = \exp(\log\pi_\theta(y_i)-\log\pi_{\mathrm{old}}(y_i))
Li=−min⁡(ρiAi,clip⁡(ρi,1−ϵ,1+ϵ)Ai)L_i = -\min(\rho_i A_i,\operatorname{clip}(\rho_i,1-\epsilon,1+\epsilon)A_i)
k3=(u−1)−log⁡u,u=π0(yi)/πθ(yi)k_3 = (u-1)-\log u,\quad u=\pi_0(y_i)/\pi_\theta(y_i)
AiRLOO=ri−1G−1∑j≠irjA_i^{\mathrm{RLOO}} = r_i - \frac{1}{G-1}\sum_{j\ne i} r_j

Read Chapter 37

38 Distillation & Reasoning Models

Key equations
qT=softmax⁡(zt/T),pT=softmax⁡(zs/T)\vq_T = \softmax(\vz^t/T),\quad \vp_T=\softmax(\vz^s/T)
LKD=T2DKL(qT∥pT)\mathcal{L}_{\mathrm{KD}} = T^2\KL(\vq_T\Vert\vp_T)
∇zsLKD=T(pT−qT)\nabla_{\vz^s}\mathcal{L}_{\mathrm{KD}} = T(\vp_T-\vq_T)
Pbest(n,p)=1−(1−p)nP_{\mathrm{best}}(n,p)=1-(1-p)^n
Pmaj(n,p)=∑k>n/2(nk)pk(1−p)n−kP_{\mathrm{maj}}(n,p)=\sum_{k>n/2}{n\choose k}p^k(1-p)^{n-k}

Read Chapter 38

39 Decoding & Speculative Sampling

Key equations
x~t=arg max⁡ipi\tilde{x}_t = \argmax_i p_i
s(y1:t)=∑ilog⁡p(yi∣y<i)s(y_{1:t}) = \sum_i \log p(y_i \mid y_{<i})
p~i=pi1{i∈S}∑jpj1{j∈S}\tilde{p}_i = \frac{p_i\mathbf{1}\{i\in S\}}{\sum_j p_j\mathbf{1}\{j\in S\}}
a(x)=min⁡(1,p(x)q(x))a(x) = \min\left(1, \frac{p(x)}{q(x)}\right)
r(x)∝max⁡(0,p(x)−q(x))r(x) \propto \max(0, p(x) - q(x))

Read Chapter 39

40 Quantization & Serving

Key equations
s=max⁡i∣xi∣2b−1−1s = \frac{\max_i |x_i|}{2^{b-1}-1}
x^i=sqi\hat{x}_i=sq_i
XW=(Xdiag⁡(s)−1)(diag⁡(s)W)\mX\mW=(\mX\operatorname{diag}(\vs)^{-1})(\operatorname{diag}(\vs)\mW)
L(q)=(w−q)⊤H(w−q)L(\vq)=(\vw-\vq)^\T\mH(\vw-\vq)
Idecode≈2P2P+BKVI_{\text{decode}} \approx \frac{2P}{2P+B_{\mathrm{KV}}}

Read Chapter 40

41 Training at Scale

Key equations
Adam bytes/param=2+2+4+4+4=16\text{Adam bytes/param}=2+2+4+4+4=16
ring bytes=2n−1n size\text{ring bytes}=2\frac{n-1}{n}\,\text{size}
ZeRO-2=2P+14P/n\text{ZeRO-2}=2P+14P/n
∑rGELU⁡(XW1,r+b1,r)W2,r=HW2\sum_r \operatorname{GELU}(\mX\mW_{1,r}+\vb_{1,r})\mW_{2,r}=\mH\mW_2
bubble=p−1m+p−1\text{bubble}=\frac{p-1}{m+p-1}

Read Chapter 41

42 Tool Use & Agent Loops

Key equations
ht=(x,a1,o1,…,at−1,ot−1)h_t = (x, a_1, o_1, \ldots, a_{t-1}, o_{t-1})
at∼pθ(tool,args∣ht)a_t \sim p_\vtheta(\text{tool}, \text{args} \mid h_t)
dispatch(at)={tool(validate(args))valid,error observationinvalid\text{dispatch}(a_t) = \begin{cases} \text{tool}(\text{validate}(\text{args})) & \text{valid},\\ \text{error observation} & \text{invalid} \end{cases}
stop∈{final,  budget,  error}\text{stop} \in \{\text{final},\; \text{budget},\; \text{error}\}

Read Chapter 42

43 Retrieval, Memory, Planning & Evaluation

Key equations
s(q,d)=eq⊤ed∥eq∥2∥ed∥2s(q, d) = \frac{\ve_q^\T \ve_d}{\lVert \ve_q\rVert_2\lVert \ve_d\rVert_2}
RAG(x)=LLM(x,d(1),…,d(k))\text{RAG}(x) = \text{LLM}(x, d_{(1)}, \ldots, d_{(k)})
pass@⁡^k=1−(n−ck)(nk)\widehat{\operatorname{pass@}}k = 1 - \frac{\binom{n-c}{k}}{\binom{n}{k}}
E[pass@⁡^k]=1−(1−p)k\E[\widehat{\operatorname{pass@}}k] = 1 - (1-p)^k
cost=∑itokensi⋅pricei+tool costi\text{cost} = \sum_i \text{tokens}_i \cdot \text{price}_i + \text{tool cost}_i

Read Chapter 43

44 Capstone: An LLM End to End

Key equations
N≈L(2d2+2dHkvdh+8d2)+VdN \approx L\big(2d^2 + 2 d H_\text{kv} d_h + 8d^2\big) + V d
C≈6ND,Dopt≈20N,Nopt≈C/120C \approx 6ND, \qquad D_\text{opt} \approx 20N, \qquad N_\text{opt} \approx \sqrt{C / 120}
Mtrain≈16N bytes,MKV=2LHkvdhT⋅bytesM_\text{train} \approx 16N \text{ bytes}, \qquad M_\text{KV} = 2 L H_\text{kv} d_h T \cdot \text{bytes}

Read Chapter 44

A Notation & Shapes

Key equations
Aˉ=∂L∂A has the shape of A,xˉ=J⊤yˉ\bar{\mA} = \frac{\partial L}{\partial \mA} \text{ has the shape of } \mA, \qquad \bar{\vx} = \mJ^\T \bar{\vy}
Y=XW+b  ⟹  Xˉ=YˉW⊤,Wˉ=X⊤Yˉ,bˉ=∑iYˉi,:\mY = \mX \mW + \vb \;\Longrightarrow\; \bar{\mX} = \bar{\mY} \mW^\T,\quad \bar{\mW} = \mX^\T \bar{\mY},\quad \bar{\vb} = \textstyle\sum_i \bar{\mY}_{i,:}

Read Appendix A

B NumPy for Deep Learning

Key equations

Broadcasting: align shapes from the right; sizes must match or be 1. The gradient of a broadcast input is the upstream gradient summed over the stretched axes.

logsumexp⁡(z)=m+log⁡∑iezi−m,m=max⁡izi\logsumexp(\vz) = m + \log \sum_i e^{z_i - m}, \quad m = \max_i z_i
max⁡izi≤logsumexp⁡(z)≤max⁡izi+log⁡n\max_i z_i \le \logsumexp(\vz) \le \max_i z_i + \log n
log⁡(1+ex)=max⁡(x,0)+log⁡(1+e−∣x∣)\log(1 + e^{x}) = \max(x, 0) + \log(1 + e^{-|x|})
f′(x)≈f(x+h)−f(x−h)2h,error=O(h2)+O(ε/h)f'(x) \approx \frac{f(x+h) - f(x-h)}{2h}, \qquad \text{error} = O(h^2) + O(\varepsilon / h)

Machine epsilon: float32 2−232^{-23}, bfloat16 2−72^{-7}, float16 2−102^{-10}, float64 2−522^{-52}.

Read Appendix B

C Matrix Calculus Cookbook

Key equations
xˉi=∑jyˉj∂yj∂xi\bar{x}_i = \sum_j \bar{y}_j\frac{\partial y_j}{\partial x_i}
Y=AB,Aˉ=YˉB⊤,Bˉ=A⊤YˉY=AB,\qquad \bar{A}=\bar{Y}B^\T,\quad \bar{B}=A^\T\bar{Y}
xˉ=y⊙(yˉ−(yˉ⊤y)1),y=softmax⁡(x)\bar{\vx}=\vy\odot(\bar{\vy}-(\bar{\vy}^\T\vy)\one),\quad \vy=\softmax(\vx)
Zˉ=softmax⁡(Z)−onehot⁡(t)B\bar{Z}=\frac{\softmax(Z)-\operatorname{onehot}(t)}{B}
O=softmax⁡(QK⊤/d)VO=\softmax(QK^\T/\sqrt d)V

Read Appendix C