Understanding a deep-learning model can mean examining a vast network of internal processing nodes. Jacob Andreas, a newly tenured MIT associate professor, is pursuing another approach: training AI systems to describe what they have learned in terms people can understand.

Andreas works in MIT’s Department of Electrical Engineering and Computer Science and is a member of its Computer Science and Artificial Intelligence Laboratory. His group’s broader aim is to help people build more accurate, robust models suited to their intended uses. He argues that the complexity of neural computation calls for tools that translate it into a form humans can interpret.

Describing what a network has learned

One strand of the group’s work examines the algorithms large language models use to learn and process information. The researchers have developed techniques that train language models to explain the workings of other neural networks, with the aim of helping researchers anticipate how a trained model might behave outside the lab.

The techniques can give a natural-language description to each information-processing node in a large network. In an image-processing system, for example, a description might identify a node that detects patches of grass near animals. Andreas says such descriptions provide a starting point for broader questions about how models represent meaning.

A second strand focuses on algorithms for training machine-learning models and predicting how particular datasets or examples affect their behavior. Andreas says the field still lacks a good scientific theory for predicting what will happen when a model is trained on a given dataset.

He expects interpretability to become more important as researchers seek to make advanced systems, including AI agents, safe enough for high-stakes deployment. He also points to models intended to assist scientific discovery: they may be trained on large quantities of laboratory data that scientists do not yet understand well. Getting such models to express knowledge that has not been explicitly written down will be both a challenge and an opportunity, he says.

From translation research to teaching

Andreas’s path into language research began at Columbia University, where he studied computer science and contacted faculty working on AI. Natural language processing researchers responded first. In Michael Collins’s lab, his undergraduate thesis adapted statistical approaches to representing meaning for use in AI translation systems.

After earning a master’s degree at Cambridge University, he pursued graduate research at the University of California at Berkeley with computer science professor Dan Klein. His studies coincided with a shift in language processing from statistical methods toward deep learning. Collaborations with computer vision researchers explored systems that could generate language by learning to understand visual information.

His Berkeley thesis examined neural module networks: small components within a larger AI system that perform tasks such as locating objects or logical reasoning. Related exploratory work on interpretability and descriptions of models’ internal states later helped shape his MIT group’s focus. He joined the MIT faculty in 2019 after a year at the startup Semantic Machines.

Rapid changes in AI also affect the undergraduate and graduate natural language processing courses Andreas co-teaches. He says instructors redesign them each year to keep pace with developments, while large language models present a challenge because they could complete assignments intended for students. He encourages students to develop a deep understanding of the technology’s foundations.