9.520/6.7910: Statistical Learning Theory and Applications Fall 2024: Course Syllabus


Class Date Title Instructor(s)
Class 01 Thu Sep 05 Course Outline + Statistical Machine Learning TP + LR
Class 02 Tue Sep 10 Regularized Least Squares LR
Class 03 Thu Sep 12 Logistic Regression and SGD LR
Class 04 Tue Sep 17 Implicit Regularization LR
Class 05 Thu Sep 19 Neural Networks - Introduction LR
Class 06 Tue Sep 24 Kernels and Feature Maps LR
Class 07 Thu Sep 26

Kernel Regression

Class 08 Tue Oct 01

Statistical Learning Theory

Class 09 Thu Oct 03 Stability  LR
Class 10 Tue Oct 08 Deep Learning: a special case of classical theory? TP
Class 11 Thu Oct 10 Deep Learning: Universal Approximation TP
Tuesday, October 15 - No Class - Student Holiday
Class 12 Thu Oct 17 Deep Learning: Approximation - Compositional Sparsity TP
Class 13 Tue Oct 22

Deep Learning: Optimization 

Class 14 Thu Oct 24

Deep Learning: Optimization - Neural Collapse

Class 15 Tue Oct 29 Deep Learning: Optimization - SGD Noise+Low Rank TP
Class 16 Thu Oct 31 Deep Learning: Generalization TP
Class 17 Tue Nov 05 Deep Learning: Transfer Learning 


Class 18 Thu Nov 07 Deep Learning: Transformers TP
Class 19 Tue Nov 12 Deep Learning: training samples from the "human distribution" ML
Class 20 Thu Nov 14 Statistical learning and scientific models in cognitive neuroscience  ML
Class 21 Tue Nov 19 Guest Lecture TBD
Class 22 Thu Nov 21 Guest Lecture TBD
Class 23 Thu Nov 26 Guest Lecture TBD
Thursday, November 28 - No Class -Thanksgiving
Class 24 Tue Dec 03 Guest Lecture TP
Class 25 Thu Dec 05 Guest Lecture TBD
Class 26 Tue Dec 05 Guest Lecture TBD

Reading List

Notes covering the classes will be provided in the form of independent chapters of a book currently in draft format. Additional information will be given through the slides associated with classes (where applicable). The books/papers listed below are useful general reference reading, especially from the theoretical viewpoint. A list of additional suggested readings will also be provided separately for each class.

Book (draft)

  • L. Rosasco and T. Poggio, Machine Learning: a Regularization Approach, MIT-9.520 Lectures Notes, Manuscript, Dec. 2017 (provided).

Primary References

Resources and links