Follow the link for each class to find a detailed description, suggested readings, and class slides. Some of the later classes may be subject to reordering or rescheduling.

Class	Date	Title	Instructor(s)
Class 01	Thu Sep 05	The Course at a Glance	TP
Class 02	Tue Sep 10	Statistical Learning Setting	LR
Class 03	Thu Sep 12	Regularized Least Squares	LR
Class 04	Tue Sep 17	Features and Kernels	LR
Class 05	Thu Sep 19	Logistic Regression and Support Vector Machines	LR
Class 06	Tue Sep 24	Learning with Stochastic Gradients	LR
Class 07	Thu Sep 26	Implicit Regularization	LR
Class 08	Tue Oct 01	Large Scale Learning by Sketching	LR
Class 09	Thu Oct 03	Sparsity Based Regularization	LR
Class 10	Tue Oct 08	Unsupervised learning of data representation	LR
Class 11	Thu Oct 10	Neural networks aka deep learning	LR
Tue Oct 15 – Columbus Day
Class 12	Thu Oct 17	Statistical Learning I	AR
Class 13	Tue Oct 22	Statistical Learning II	AR
Class 14	Thu Oct 24	ERM, Uniform Convergence	AR
Class 15	Tue Oct 29	Sample Complexity via Rademacher Averages	AR
Class 16	Thu Oct 31	Margin Analysis for Classification	AR
Class 17	Tue Nov 05	Local Methods	AR
Class 18	Thu Nov 07	Sample Compression, Stability	AR
Class 19	Tue Nov 12	Privacy and Information-Theoretic Stability	AR
Class 20	Thu Nov 14	Online Prediction	AR
Class 21	Tue Nov 19	Sample complexity of Neural Networks	AR
Class 22	Thu Nov 21	Interpolation, Overfitting, and Neural Networks	AR
Class 23	Tue Nov 26	Deep Learning Theory: Approximation	TP
Thu Nov 28 – Thanksgiving
Class 24	Tue Dec 03	Guest lecture	A. Madry
Class 25	Thu Dec 05	Deep Learning Theory: Optimization and Dynamics	TP
Class 26	Tue Dec 10	Learning Theory: Rebooting the foundations	TP
Wed Dec 11 – Project reports due

Reading List

Notes covering the classes will be provided in the form of independent chapters of a book currently in draft format. Additional information will be given through the slides associated with classes (where applicable). The books/papers listed below are useful general reference reading, especially from the theoretical viewpoint. A list of additional suggested readings will also be provided separately for each class.

Book (draft)

• L. Rosasco and T. Poggio, Machine Learning: a Regularization Approach, MIT-9.520 Lectures Notes, Manuscript, Dec. 2017 (provided).

Primary References

• S. Shalev-Shwartz and S. Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
• T. Hastie, R. Tibshirani and J. Friedman. The Elements of Statistical Learning. 2nd Ed., Springer, 2009.
• I. Steinwart and A. Christmann. Support Vector Machines. Springer, 2008.
• O. Bousquet, S. Boucheron and G. Lugosi. Introduction to Statistical Learning Theory. Advanced Lectures on Machine Learning, LNCS 3176, pp. 169-207. (Eds.) Bousquet, O., U. von Luxburg and G. Ratsch, Springer, 2004.
• N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, 2000.
• F. Cucker and S. Smale. On The Mathematical Foundations of Learning. Bulletin of the American Mathematical Society, 2002.
• F. Cucker and D-X. Zhou. Learning theory: an approximation theory viewpoint. Cambridge Monographs on Applied and Computational Mathematics. Cambridge University Press, 2007.
• L. Devroye, L. Gyorfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Springer, 1997.
• T. Evgeniou, M. Pontil and T. Poggio. Regularization Networks and Support Vector Machines. Advances in Computational Mathematics, 2000.
• T. Poggio and S. Smale. The Mathematics of Learning: Dealing with Data. Notices of the AMS, 2003.
• V. N. Vapnik. Statistical Learning Theory. Wiley, 1998.
• V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, 2000.
• S. Villa, L. Rosasco, T. Poggio. On Learnability, Complexity and Stability. Empirical Inference: Festschrift in Honor of Vladimir N. Vapnik, Chapter 7, pp. 59-70, Springer-Verlag, 2013.
• T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda, and Q. Liao. Why and When can Deep-but not Shallow-Networks Avoid the Curse of Dimensionality: A Review. International Journal of Automation and Computing, 1-17, 2017.
• Y. LeCun, Y. Bengio and G. Hinton, Deep Learning, Nature, 436-444, 2015.
   

Resources and links

• Machine Learning 2017-2018. University of Genoa, graduate ML course.
• L. Rosasco, Introductory Machine Learning Notes, University of Genoa, ML 2016/2017 lectures notes, Oct. 2016.