9.520 F2020 Course Syllabus

Follow the link for each class to find a detailed description, suggested readings, and class slides. Some of the later classes may be subject to reordering or rescheduling.

Class Date Title Instructor(s)
Class 01 Tue Sep 01 The Course at a Glance TP
Class 02 Thu Sep 03 Statistical Learning Setting LR
Mon Sep 07 - Labor Day
Class 03 Tue Sep 08 Regularized Least Squares LR
Class 04 Thu Sep 10 Features and Kernels LR
Class 05 Tue Sep 15 Logistic Regression and Support Vector Machines LR
Class 06 Thu Sep 17 Learning with Stochastic Gradients LR
Class 07 Tue Sep 22

Implicit Regularization with linear networks 

LR
Class 08 Thu Sep 24

Learning with Random Features

LR
Class 09 Tue Sep 29 Approximation and Estimation Error LR
Class 10 Thu Oct 01 Stability of Ridge Regression LR
Class 11 Tue Oct 06

Condition Number, Overparameterization Puzzles,

 Stability of Ridgeless Regression

TP + AR
Class 12 Thu Oct 08 Introduction to Deep Networks LR
Mon Oct 12 - Columbus Day
Class 13 Thu Oct 15 Deep Learning Theory: Approximation TP
Class 14 Tue Oct 20

Deep Learning: Optimization and Dynamics

(exponential loss functions)

TP+AB
Class 15 Thu Oct 22

Deep Learning: Optimization and Generalization puzzles

TP
Class 16 Tue Oct 27 Group Invariants in Vision TP+Fabio Anselmi
Class 17 Thu Oct 29 Invariance, Neurons, Synaptic Plasticity, Development TP+Fabio Anselmi
Class 18 Tue Nov 03 Neural Networks and the Ventral Stream TP+ Thomas Serre + Gabriel Kreiman
Class 19 Thu Nov 05 Neural Networks and the Ventral Stream TP+ Thomas Serre + Gabriel Kreiman
Class 20 Tue Nov 10 Loose Ends Staff 
Wed Nov 11 - Veterans Day
Class 21 Thu Nov 12 Programming brain routines with recurrent networks Shimon Ullman
Class 22 Tue Nov 17 Guest Lecture - Adversarial Attacks, Robust Training Aleksander Madry
Class 23 Thu Nov 19 Guest Lecture - Bias, Fairness, Accountability Timnit Gebru
Sat Nov 21 - Sun Nov 29 - Thanksgiving Break
Class 24 Tue Dec 01 Guest Lecture - Neuronal Ensembles Christos Papadimitriou+Santosh Vempala
Class 25 Thu Dec 03 Guest Lecture - Neural Collapse in Deep Learning David Donoho + Vardan Papyan
Class 26 Tue Dec 08 Guest Lecture - Computational Complexity of Gradient Descent Shai Shalev Schwartz
Wed Dec 09 - Project presentations

Reading List

Notes covering the classes will be provided in the form of independent chapters of a book currently in draft format. Additional information will be given through the slides associated with classes (where applicable). The books/papers listed below are useful general reference reading, especially from the theoretical viewpoint. A list of additional suggested readings will also be provided separately for each class.

Book (draft)

  • L. Rosasco and T. Poggio, Machine Learning: a Regularization Approach, MIT-9.520 Lectures Notes, Manuscript, Dec. 2017 (provided).

Primary References

Resources and links