Follow the link for each class to find a detailed description, suggested readings, and class slides. Some of the later classes may be subject to reordering or rescheduling.

Class	Date	Title	Instructor(s)
Class 01	Tue Sep 01	The Course at a Glance	TP
Class 02	Thu Sep 03	Statistical Learning Setting	LR
Mon Sep 07 – Labor Day
Class 03	Tue Sep 08	Regularized Least Squares	LR
Class 04	Thu Sep 10	Features and Kernels	LR
Class 05	Tue Sep 15	Logistic Regression and Support Vector Machines	LR
Class 06	Thu Sep 17	Learning with Stochastic Gradients	LR
Class 07	Tue Sep 22	Implicit Regularization with linear networks	LR
Class 08	Thu Sep 24	Learning with Random Features	LR
Class 09	Tue Sep 29	Approximation and Estimation Error	LR
Class 10	Thu Oct 01	Stability of Ridge Regression	LR
Class 11	Tue Oct 06	Condition Number, Overparameterization Puzzles,  Stability of Ridgeless Regression	TP + AR
Class 12	Thu Oct 08	Introduction to Deep Networks	LR
Mon Oct 12 – Columbus Day
Class 13	Thu Oct 15	Deep Learning Theory: Approximation	TP
Class 14	Tue Oct 20	Deep Learning: Optimization and Dynamics (exponential loss functions)	TP+AB
Class 15	Thu Oct 22	Deep Learning: Optimization and Generalization puzzles	TP
Class 16	Tue Oct 27	Group Invariants in Vision	TP+Fabio Anselmi
Class 17	Thu Oct 29	Invariance, Neurons, Synaptic Plasticity, Development	TP+Fabio Anselmi
Class 18	Tue Nov 03	Neural Networks and the Ventral Stream	TP+ Thomas Serre + Gabriel Kreiman
Class 19	Thu Nov 05	Neural Networks and the Ventral Stream	TP+ Thomas Serre + Gabriel Kreiman
Class 20	Tue Nov 10	Loose Ends	Staff
Wed Nov 11 – Veterans Day
Class 21	Thu Nov 12	Programming brain routines with recurrent networks	Shimon Ullman
Class 22	Tue Nov 17	Guest Lecture – Adversarial Attacks, Robust Training	Aleksander Madry
Class 23	Thu Nov 19	Guest Lecture – Bias, Fairness, Accountability	Timnit Gebru
Sat Nov 21 – Sun Nov 29 – Thanksgiving Break
Class 24	Tue Dec 01	Guest Lecture – Neuronal Ensembles	Christos Papadimitriou+Santosh Vempala
Class 25	Thu Dec 03	Guest Lecture – Neural Collapse in Deep Learning	David Donoho + Vardan Papyan
Class 26	Tue Dec 08	Guest Lecture – Computational Complexity of Gradient Descent	Shai Shalev Schwartz
Wed Dec 09 – Project presentations

Reading List

Notes covering the classes will be provided in the form of independent chapters of a book currently in draft format. Additional information will be given through the slides associated with classes (where applicable). The books/papers listed below are useful general reference reading, especially from the theoretical viewpoint. A list of additional suggested readings will also be provided separately for each class.

Book (draft)

• L. Rosasco and T. Poggio, Machine Learning: a Regularization Approach, MIT-9.520 Lectures Notes, Manuscript, Dec. 2017 (provided).

Primary References

• S. Shalev-Shwartz and S. Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
• T. Hastie, R. Tibshirani and J. Friedman. The Elements of Statistical Learning. 2nd Ed., Springer, 2009.
• I. Steinwart and A. Christmann. Support Vector Machines. Springer, 2008.
• O. Bousquet, S. Boucheron and G. Lugosi. Introduction to Statistical Learning Theory. Advanced Lectures on Machine Learning, LNCS 3176, pp. 169-207. (Eds.) Bousquet, O., U. von Luxburg and G. Ratsch, Springer, 2004.
• N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, 2000.
• F. Cucker and S. Smale. On The Mathematical Foundations of Learning. Bulletin of the American Mathematical Society, 2002.
• F. Cucker and D-X. Zhou. Learning theory: an approximation theory viewpoint. Cambridge Monographs on Applied and Computational Mathematics. Cambridge University Press, 2007.
• L. Devroye, L. Gyorfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Springer, 1997.
• T. Evgeniou, M. Pontil and T. Poggio. Regularization Networks and Support Vector Machines. Advances in Computational Mathematics, 2000.
• T. Poggio and S. Smale. The Mathematics of Learning: Dealing with Data. Notices of the AMS, 2003.
• V. N. Vapnik. Statistical Learning Theory. Wiley, 1998.
• V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, 2000.
• S. Villa, L. Rosasco, T. Poggio. On Learnability, Complexity and Stability. Empirical Inference: Festschrift in Honor of Vladimir N. Vapnik, Chapter 7, pp. 59-70, Springer-Verlag, 2013.
• T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda, and Q. Liao. Why and When can Deep-but not Shallow-Networks Avoid the Curse of Dimensionality: A Review. International Journal of Automation and Computing, 1-17, 2017.
• T. Poggio, A. Banburski, Q. Liao. Theoretical Issues in Deep Networks. PNAS, 1-7, 2020.
• Y. LeCun, Y. Bengio and G. Hinton, Deep Learning, ​Nature, 436-444, 2015.

Resources and links

• Machine Learning 2017-2018. University of Genoa, graduate ML course.
• L. Rosasco, Introductory Machine Learning Notes, University of Genoa, ML 2016/2017 lectures notes, Oct. 2016.