Adam, AdamW, and the LLM Recipe

g = 1, β1 = 0.9, β2 = 0.999, η = 10⁻³: bias-corrected m̂ = 1, v̂ = 1; θ moves 1 → 0.999. AdamW decay is decoupled — not L2 inside Adam.

This Optimization lesson is part of a paid plan. Sign in with a plan that includes this course to read it.