The Science of Intelligence

Preliminary list of Projects

1. Consider multiplicative or additive regularization or no regularization: visualize different landscape on same problem like CIFAR or binary Cifar. Are there local minima? Global Minima? Contact Tommy Poggio tp@csail.mit.edu.
2. Show empirically that to get Neural Collapse, regularization is needed for the square loss but it is not needed for exponential loss functions. Contact Tommy Poggio tp@csail.mit.edu.
3. Read https://arxiv.org/pdf/2407.13841. I think this paper — if correct — may help solve an old puzzle in AI (which is not mentioned in the paper). I will reveal it in class. Contact Tommy Poggio tp@csail.mit.edu.
4. Brain TreeBank data exploration: the InfoLab has collected 40 hours of intracranial neural activity, recorded while subjects watched movies. For each movie, we have the dialogue and various visual features aligned to the neural recordings. This project will focus on annotating a new feature in the movie, e.g., facial expression, and then training a decoder to localize processing of this feature. There are also annotated features, for which no decoding has yet been run, e.g., labeled scene decoding. See the BrainBERT paper for an example of decoding with this data. Contact Christopher Wang czw@mit.edu.
5. Brain and neural network alignment: this project will use the same data as above, but will focus on alignment with a neural network of the student’s choosing. See this paper for an example of using neural network features to encode brain activity. Contact Vighnesh Subramaniam vsub851@mit.edu.
6. The Natural Scenes Dataset (NSD) contains fMRI recordings from subjects who were shown naturalistic photos. Recently, methods have been proposed to reconstruct these images from the fMRI readings with seemingly high quality. Some open questions remain: which features of these images are most reconstructable? Which are not? Contact David Mayo dmayo2@mit.edu and Christopher Wang czw@mit.edu.
7. We may have reached a turning point where determining what these models cannot do is easier to isolate than closing the space of things they can accomplish. Decades ago, this pursuit would have been impractical, as these models failed far more often than they succeeded. But things have changed. And with that, we propose looking more closely at what these models get wrong and what causes lie therein. We propose gathering high-loss/high-error examples from top-tier language models and analyzing what commonalities and differences are shared amongst them. Contact Brian Cheung cheungb@mit.edu.
8. The field has found that Artificial Intelligence models are becoming increasingly similar to their Naturally Intelligent counterparts, at the structural (https://arxiv.org/abs/1611.09430), representational (https://arxiv.org/abs/2302.06677), and behavioral levels. Why is this happening? We have one hypothesis (https://arxiv.org/abs/2405.07987). But how far does this hypothesis go? We will explore whether platonic representations are present far beyond the modalities of language and vision, potentially at the level of molecules. Contact Brian Cheung cheungb@mit.edu.
9. In deep learning, certain tasks like image classification use specific “recipes” for model design i.e. particular model architectures that are known to work like ConvNets. In this project, we aim to expand architecture design to include “untrainable” networks for these tasks such as MLPs for image classification, and designed a training technique based on representational similarity between a guide network and a target network similar to how networks are compared to the brain. We have applied this approach on several networks and are looking to expand to more ambitious architectures such as a CNN language model/RNN image classifier and perform new kinds of analyses to understand what our approach can tell us about neural network design. optimization, and model initialization. Familiarity with pytorch is recommended but not required. Contact Vighnesh Subramaniam vsub851@mit.edu.
10. Dendrites facilitate Grid cell formation: (Most biologically mechanistic) Population activity of Grid cells in the medial entorhinal cortex has a toroidal topology (ref). It has not been shown how the necessary connectivity matrix could arise biologically, and solutions that propose intermingled ring networks seem particularly unlikely to form from known biological synaptic plasticity rules. We propose that the two dimensional toroidal manifold can be easily formed in a biological way with the inclusion of spatially localized dendrites, dendritic nonlinearities (ref), and a dendritic hebbian learning rule. The goal of this project would be to show this by coding up the network structure and learning rule, and allowing it to evolve with simulated motor and visual inputs in a simulated environment. Contact Gregg Heller greggh@mit.edu.
11. Grid cells as a template for the cortical microcircuit: (Most intelligence oriented) One of the mysteries of mammalian cortex is this: different cortical areas represent different information with seemingly different goals (e.g. Lower level sensory processing vs language production vs planning and context dependent, goal oriented action) yet the anatomical structure is nearly identical across the entire cortical sheet. This suggests that the same circuit can be used for all these purposes with only minor modifications. It has been proposed conceptually that the grid cell attractor circuit could underlie all of perception and cognition (ref). To this end, we aim to show that a biologically plausible grid cell circuit (ref1, ref2) can be used to process object motion in visual scenes, providing a new circuit mechanism for the cortical predictive processing hypothesis. We contemplate the application of such a circuit to something like language production and processing. Contact Gregg Heller greggh@mit.edu.
12. Cell types from in-vivo ephys data: (most mahcine learning-y) The cortex contains multiple inhibitory subclasses (VIP, SST, PV etc) which have different connectivity to eachother and to excitatory neurons. Understanding cortical computation will likely require recording from these populations, however it is currently difficult to identify SST and VIP cells in electrical recordings from living brains. Progress has been made to identify SST cells  (ref) using the unit mean waveform in addition to other features. However attempts to date incorporate only one dimension of available waveform data (voltage on peak channel through time) and neglect the spatial sampling of modern probes (X position x Z position x T). We believe that incorporating the 3 dimensional waveform should make classification of VIP cells possible due to their uniqe narrow profile and downward reaching dendrites (fig 7 of ref). This project aims to train a classifier that outperforms Fig6 of (ref) using the same data (actually much more is available now from same source experiment). Contact Gregg Heller greggh@mit.edu.
13. Recurrent neural networks and the brain: This project will explore the similarities and differences between recurrent neural network models and neural recordings using differentiable similarity measures, and follow-up on work from this paper: https://arxiv.org/abs/2407.07059  Contact Chris Cueva at cjcueva@mit.edu
14. Does enhancing the NEMO model of neuron assemblies yield new computational power? The NEMO model (or the Assembly Calculus model) introduced by Papadimitriou and Vempala has emerged as a candidate mathematical abstraction of neural computation (in particular as a model of cortex). The model takes a very small, minimalistic set of neurally plausible assumptions, and constructs a model in which densely interconnected, co-firing sets of neurons (“assemblies”) naturally emerge; a series of papers has shown that this system can be used to perform various computations, including some learning tasks and language parsing! The question is whether strengthening this model with additional biological realism can yield a stronger model (this can be demonstrated mathematically or through simulation), or whether such an enhanced model might be more efficiently simulateable or applicable for AI; possible enhancements to study might be inhbition, distinct neuron types, dendritic computation, or a more sophisticated model of plasticity. Contact Danny Mitropolsky at mitropol@mit.edu.
15. Language with neuron assemblies. A series of papers have studied how various linguistic subfunctions, specifically parsing and the acqusition of simple “concrete” nouns and verbs, can be achieved in the NEMO model of brain computation; these results both demonstrate the “viability” of NEMO as a model of the brain, and are also theories of language in the brain. Many avenues are made open by this work: coming up with an algorithm for learning more word representations with some structure, syntactic acqusition, or looking at phonology are all interesting directions. Contact Danny Mitropolsky at mitropol@mit.edu.

• Starting-point papers (for Danny’s projects):
• Intro paper to the NEMO model and the emergence of neuronal assemblies:”Brain computation by assemblies of neurons”
• https://www.pnas.org/doi/full/10.1073/pnas.2001893117
• On algorithms for a particular class of problems (learning distribution of a discrete, conditional random variable) in the NEMO model, and minimal assumptions:
• “Coin-Flipping In The Brain: Statistical Learning with Neuronal Assemblies”
• https://arxiv.org/abs/2406.07715
• On parsing in the NEMO model:
• “A Biologically Plausible Parser”
• https://arxiv.org/abs/2108.02189
• “Center-Embedding and Constituency in the Brain and a New Characterization of Context-Free Languages”
• https://aclanthology.org/2022.naloma-1.4/
• “The Architecture of a Biologically Plausible Language Organ”
• https://arxiv.org/abs/2306.15364
• For other background / comparison it could be good to skim through older “classic” theoretical literature on assemblies:
• Marr 1971 Simple memory: a theory for archicortex for Marr’s computational models of cortex and hippocampus (recurrent networks implementing pattern completion)
• Hopfield 1982 for Hopfield networks / models of attractors in recurrent networks
• Hopfield networks as a computational model: Hopfield and Tank 1985 “Neural” computation of decisions in optimization problems, and  J.J. Hopfield, D.W. Tank 1986 Computing with neural circuits: A model
• Abeles’ Corticonics model / synfire chains (1991 Corticonics