Follow the link for each class to find a detailed description, suggested readings, and class slides. Some of the later classes may be subject to reordering or rescheduling.

Class | Date | Title | Instructor(s) |
---|---|---|---|

Class 01 | Tue Sep 01 | The Course at a Glance | TP |

Class 02 | Thu Sep 03 | Statistical Learning Setting | LR |

Mon Sep 07 - Labor Day | |||

Class 03 | Tue Sep 08 | Regularized Least Squares | LR |

Class 04 | Thu Sep 10 | Features and Kernels | LR |

Class 05 | Tue Sep 15 | Logistic Regression and Support Vector Machines | LR |

Class 06 | Thu Sep 17 | Learning with Stochastic Gradients | LR |

Class 07 | Tue Sep 22 | LR | |

Class 08 | Thu Sep 24 | LR | |

Class 09 | Tue Sep 29 | Approximation and Estimation Error | LR |

Class 10 | Thu Oct 01 | Stability of Ridge Regression | LR |

Class 11 | Tue Oct 06 | TP + AR | |

Class 12 | Thu Oct 08 | Introduction to Deep Networks | LR |

Mon Oct 12 - Columbus Day | |||

Class 13 | Thu Oct 15 | Deep Learning Theory: Approximation | TP |

Class 14 | Tue Oct 20 |
Deep Learning: Optimization and Dynamics (exponential loss functions) |
TP+AB |

Class 15 | Thu Oct 22 |
Deep Learning: Optimization and Generalization puzzles |
TP |

Class 16 | Tue Oct 27 | Group Invariants in Vision | TP+Fabio Anselmi |

Class 17 | Thu Oct 29 | Invariance, Neurons, Synaptic Plasticity, Development | TP+Fabio Anselmi |

Class 18 | Tue Nov 03 | Neural Networks and the Ventral Stream | TP+ Thomas Serre + Gabriel Kreiman |

Class 19 | Thu Nov 05 | Neural Networks and the Ventral Stream | TP+ Thomas Serre + Gabriel Kreiman |

Class 20 | Tue Nov 10 | Loose Ends | Staff |

Wed Nov 11 - Veterans Day | |||

Class 21 | Thu Nov 12 | Programming brain routines with recurrent networks | Shimon Ullman |

Class 22 | Tue Nov 17 | Guest Lecture - Adversarial Attacks, Robust Training | Aleksander Madry |

Class 23 | Thu Nov 19 | Guest Lecture - Bias, Fairness, Accountability | Timnit Gebru |

Sat Nov 21 - Sun Nov 29 - Thanksgiving Break | |||

Class 24 | Tue Dec 01 | Guest Lecture - Neuronal Ensembles | Christos Papadimitriou+Santosh Vempala |

Class 25 | Thu Dec 03 | Guest Lecture - Neural Collapse in Deep Learning | David Donoho + Vardan Papyan |

Class 26 | Tue Dec 08 | Guest Lecture - Computational Complexity of Gradient Descent | Shai Shalev Schwartz |

Wed Dec 09 - Project presentations |

**Reading List**

Notes covering the classes will be provided in the form of independent chapters of a book currently in draft format. Additional information will be given through the slides associated with classes (where applicable). The books/papers listed below are useful general reference reading, especially from the theoretical viewpoint. A list of additional suggested readings will also be provided separately for each class.

**Book (draft)**

- L. Rosasco and T. Poggio,
**Machine Learning: a Regularization Approach, MIT-9.520 Lectures Notes**, Manuscript, Dec. 2017**(provided)**.

**Primary References**

- S. Shalev-Shwartz and S. Ben-David.
**Understanding Machine Learning: From Theory to Algorithms.**Cambridge University Press, 2014. - T. Hastie, R. Tibshirani and J. Friedman.
**The Elements of Statistical Learning**. 2nd Ed., Springer, 2009. - I. Steinwart and A. Christmann.
**Support Vector Machines.**Springer, 2008. - O. Bousquet, S. Boucheron and G. Lugosi.
**Introduction to Statistical Learning Theory**. Advanced Lectures on Machine Learning, LNCS 3176, pp. 169-207. (Eds.) Bousquet, O., U. von Luxburg and G. Ratsch, Springer, 2004. - N. Cristianini and J. Shawe-Taylor.
**An Introduction to Support Vector Machines and Other Kernel-based Learning Methods.**Cambridge University Press, 2000. - F. Cucker and S. Smale.
**On The Mathematical Foundations of Learning**. Bulletin of the American Mathematical Society, 2002. - F. Cucker and D-X. Zhou.
**Learning theory: an approximation theory viewpoint.**Cambridge Monographs on Applied and Computational Mathematics. Cambridge University Press, 2007. - L. Devroye, L. Gyorfi, and G. Lugosi.
**A Probabilistic Theory of Pattern Recognition.**Springer, 1997. - T. Evgeniou, M. Pontil and T. Poggio.
**Regularization Networks and Support Vector Machines.**Advances in Computational Mathematics, 2000. - T. Poggio and S. Smale.
**The Mathematics of Learning: Dealing with Data.**Notices of the AMS, 2003. - V. N. Vapnik.
**Statistical Learning Theory.**Wiley, 1998. - V. N. Vapnik.
**The Nature of Statistical Learning Theory.**Springer, 2000. - S. Villa, L. Rosasco, T. Poggio.
**On Learnability, Complexity and Stability**. Empirical Inference: Festschrift in Honor of Vladimir N. Vapnik, Chapter 7, pp. 59-70, Springer-Verlag, 2013. - T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda, and Q. Liao.
**Why and When can Deep-but not Shallow-Networks Avoid the Curse of Dimensionality: A Review**. International Journal of Automation and Computing, 1-17, 2017. - T. Poggio, A. Banburski, Q. Liao.
**Theoretical Issues in Deep Networks.**PNAS, 1-7, 2020. - Y. LeCun, Y. Bengio and G. Hinton,
__Deep Learning,__Nature, 436-444, 2015.

**Resources and links**

- Machine Learning 2017-2018. University of Genoa, graduate ML course.
- L. Rosasco,
**Introductory Machine Learning Notes**, University of Genoa, ML 2016/2017 lectures notes, Oct. 2016.