Follow the link for each class to find a detailed description, suggested readings, and class slides. Some of the later classes may be subject to reordering or rescheduling.

Class

Class	Date	Title	Instructor(s)
Class 01	Thu Sep 08	Course Outline. Statistical Machine Learning	LR
Class 02	Tue Sep 13	Empirical Risk Minimization and Regularization for Linear Models	LR
Class 03	Thu Sep 15	Kernels and Feature Maps	LR
Class 04	Tue Sep 20	Optimization: GD and SGD. Regularization and implicit regularization	TG
Class 05	Thu Sep 22	Error Decomposition and Approximation Error	AR
Class 06	Tue Sep 27	Estimation Error and Generalization Gap	AR
Class 07	Thu Sep 29	Stability of Ridge and Ridgeless Regression	AR
Class 08	Tue Oct 04	Deep Learning Theory: Approximation	TP
Class 09	Thu Oct 06	Introduction to Deep Networks	AR
Monday 10th October – Indigenous People’s Day, Tuesday 11th October – Student Holiday
Class 10	Thu Oct 13	Deep Learning: Optimization and Dynamics	TP
Class 11	Tue Oct 18	Deep Learning: Bias towards Low Rank	TG
Class 12	Thu Oct 20	Deep Learning: Neural Collapse	AR + TG
Class 13	Tue Oct 25	Deep Learning: Generalization in Sparse Overparametrized Networks	TP
Class 14	Thu Oct 27	Group Invariance and Equivariance in Vision and Learning	Fabio Anselmi
Class 15	Tue Nov 01	Transformers	Brian Cheung
Class 16	Thu Nov 03	Neural Networks and the Ventral Stream	Thomas Serre + Gabriel Kreiman
Class 17	Tue Nov 08	Loose Ends	Staff
Class 18	Thu Nov 10	Brain and Neural Networks – Identification Problems	Brian Cheung + Yena Han
Class 19	Tue Nov 15	Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit	Benjamin Edelman
Class 20	Thu Nov 17	Optimal tradeoff approximation/generalization in Underparametrized  deep networks	Sophie Langer
Class 21	Tue Nov 22	Adversarial examples	Alexander Madry
Thursday 24th November – Thanksgiving
Class 22	Tue Nov 29	Neural Assemblies	Christos Papadimitriou +Santosh Vempala
Class 23	Thu Dec 01	Brainstorming on deep learning puzzles+projects and other thoughts	TP and AR
Class 24	Tue Dec 06	Sparsity in linear models and deep networks	Yuan Yao
Class 25	Thu Dec 08	The loss landscape of overparametrized deep nets	Yaim Cooper
Class 26	Tue Dec 13	Transformers	Tomer Ullman

Reading List

Notes covering the classes will be provided in the form of independent chapters of a book currently in draft format. Additional information will be given through the slides associated with classes (where applicable). The books/papers listed below are useful general reference reading, especially from the theoretical viewpoint. A list of additional suggested readings will also be provided separately for each class.

Book (draft)

• L. Rosasco and T. Poggio, Machine Learning: a Regularization Approach, MIT-9.520 Lectures Notes, Manuscript, Dec. 2017 (provided).

Primary References

• S. Shalev-Shwartz and S. Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
• O. Bousquet, S. Boucheron and G. Lugosi. Introduction to Statistical Learning Theory. Advanced Lectures on Machine Learning, LNCS 3176, pp. 169-207. (Eds.) Bousquet, O., U. von Luxburg and G. Ratsch, Springer, 2004.
• F. Cucker and S. Smale. On The Mathematical Foundations of Learning. Bulletin of the American Mathematical Society, 2002.
• L. Devroye, L. Gyorfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Springer, 1997.
• T. Evgeniou, M. Pontil and T. Poggio. Regularization Networks and Support Vector Machines. Advances in Computational Mathematics, 2000.
• T. Poggio and S. Smale. The Mathematics of Learning: Dealing with Data. Notices of the AMS, 2003.
• V. N. Vapnik. Statistical Learning Theory. Wiley, 1998.
• V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, 2000.
• S. Villa, L. Rosasco, T. Poggio. On Learnability, Complexity and Stability. Empirical Inference: Festschrift in Honor of Vladimir N. Vapnik, Chapter 7, pp. 59-70, Springer-Verlag, 2013.
• T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda, and Q. Liao. Why and When can Deep-but not Shallow-Networks Avoid the Curse of Dimensionality: A Review. International Journal of Automation and Computing, 1-17, 2017.
• T. Poggio, A. Banburski, Q. Liao. Theoretical Issues in Deep Networks. PNAS, 1-7, 2020.
• Y. LeCun, Y. Bengio and G. Hinton, Deep Learning, ​Nature, 436-444, 2015.

Resources and links

• Machine Learning 2017-2018. University of Genoa, graduate ML course.
• L. Rosasco, Introductory Machine Learning Notes, University of Genoa, ML 2016/2017 lectures notes, Oct. 2016.