Trust Regions and KL Balls

KL(q‖p)=0.1927… for (0.8,0.2) vs (0.5,0.5). PPO ε=0.2 clips ratio 1.3→1.2, A=2 → obj 2.4. ChatGPT-class RLHF leash; DPO folds the wall in.

This Optimization lesson is part of a paid plan. Sign in with a plan that includes this course to read it.