TP + TG Loose Ends Materials for each class – including a detailed description, suggested readings, and class slides – may be found on the class Canvas page. Some of the later classes may be subject to reordering or rescheduling.

Class	Date	Title	Instructor(s)
Class 01	Thu Sep 07	Course Outline + Statistical Machine Learning	TP + LR
Class 02	Tue Sep 12	Regularized Least Squares	LR
Class 03	Thu Sep 14	Logistic Regression and SGD	LR
Class 04	Tue Sep 19	Implicit Regularization	LR
Class 05	Thu Sep 21	Neural Networks – Introduction	LR
Class 06	Tue Sep 26	Kernels and Feature Maps	LR
Class 07	Thu Sep 28	Kernel Regression	LR
Class 08	Tue Oct 03	Statistical Learning Theory	LR
Class 09	Thu Oct 05	Stability	LR
Monday 9th October – Indigenous People’s Day, Tuesday 10th October – Student Holiday No Classes
Class 10	Thu Oct 12	Deep Learning: Tips and Tricks	AR
Class 11	Tue Oct 17	Deep Learning: Approximation – Universal Approximation	TP
Class 12	Thu Oct 19	Deep Learning: Approximation – Compositional Sparsity	TP
Class 13	Tue Oct 24	Deep Learning: Optimization	TP
Class 14	Thu Oct 26	Deep Learning: Optimization – Neural Collapse	AR
Class 15	Tue Oct 31	Deep Learning: Optimization – SGD Noise+Low Rank	TP+TG
Class 16	Thu Nov 02	Deep Learning: Generalization	TP+TG
Class 17	Tue Nov 07	Deep Learning: Transfer Learning	TG
Class 18	Thu Nov 09	Deep Learning: Trends in Architecture and Transformers	Brian Cheung
Class 19	Tue Nov 14	Deep Learning: Transformers	TP
Class 20	Thu Nov 16	Group Invariance and Equivariance in Vision and Learning	Fabio Anselmi
Class 21	Tue Nov 21	Neural Networks and the Ventral Stream	Thomas Serre + Gabriel Kreiman
Thursday 23rd November – Thanksgiving
Class 22	Tue Nov 28	Fireside on AI, LLMs and OpenAI	Alexander Madry
Class 23	Thu Nov 30	Curses and Blessings of Dimensionality	David Donoho
Class 24	Tue Dec 05	Loose Ends	TP + TG
Class 25	Thu Dec 07	On simple transformers	Alberto Bietti
Class 26	Tue Dec 12	AI and Regulations	Theos Evgeniou

Reading List

Notes covering the classes will be provided in the form of independent chapters of a book currently in draft format. Additional information will be given through the slides associated with classes (where applicable). The books/papers listed below are useful general reference reading, especially from the theoretical viewpoint. A list of additional suggested readings will also be provided separately for each class.

Book (draft)

• L. Rosasco and T. Poggio, Machine Learning: a Regularization Approach, MIT-9.520 Lectures Notes, Manuscript, Dec. 2017 (provided).

Primary References

• S. Shalev-Shwartz and S. Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
• O. Bousquet, S. Boucheron and G. Lugosi. Introduction to Statistical Learning Theory. Advanced Lectures on Machine Learning, LNCS 3176, pp. 169-207. (Eds.) Bousquet, O., U. von Luxburg and G. Ratsch, Springer, 2004.
• F. Cucker and S. Smale. On The Mathematical Foundations of Learning. Bulletin of the American Mathematical Society, 2002.
• L. Devroye, L. Gyorfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Springer, 1997.
• T. Evgeniou, M. Pontil and T. Poggio. Regularization Networks and Support Vector Machines. Advances in Computational Mathematics, 2000.
• T. Poggio and S. Smale. The Mathematics of Learning: Dealing with Data. Notices of the AMS, 2003.
• V. N. Vapnik. Statistical Learning Theory. Wiley, 1998.
• T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda, and Q. Liao. Why and When can Deep-but not Shallow-Networks Avoid the Curse of Dimensionality: A Review. International Journal of Automation and Computing, 1-17, 2017.
• T. Poggio, A. Banburski, Q. Liao. Theoretical Issues in Deep Networks. PNAS, 1-7, 2020.
• Y. LeCun, Y. Bengio and G. Hinton, Deep Learning, ​Nature, 436-444, 2015.

Resources and links

• Machine Learning 2017-2018. University of Genoa, graduate ML course.
• L. Rosasco, Introductory Machine Learning Notes, University of Genoa, ML 2016/2017 lectures notes, Oct. 2016.