Course Description
This course explores modern perspectives on generalization in machine learning, connecting classical statistical learning theory with surprising behaviors of deep and overparameterized models. We study both the theoretical foundations and empirical phenomena, including double descent, benign overfitting, and sequential analysis. Students will engage with recent research papers and carry out projects analyzing generalization in real or simulated settings.
This course is inspired by, and at times will closely follow, Surbhi Goel's course at Penn.
Lecture Notes and Optional Reading
- Losses, concentration inequalities [notes]
- Ben Recht's blog post: The Adaptivity Paradox
- SQs, PAC learning finite hypothesis classes [notes]
- Overfitting with adaptive SQs [notes]
- VC dimension and PAC learnability [notes]
Course Resources
- Reading list and presentation schedule
- Canvas (includes Courselore discussion link)
- Recordings