The research program

Understanding by building

“What I cannot create, I do not understand.” Richard Feynman

We study the mind and brain as they operate in real life. Most of what cognitive science knows comes from controlled experiments, which buy precision by narrowing the conditions under which their findings hold. Our lab takes a complementary route. We build computational agents that learn from the same everyday experience people do, and we judge them as we judge people: by what they can do in the world.

The program has three objectives that feed one another. Record: we capture human experience in dense detail as it unfolds in real life. Build: we train agents on that experience, under human constraints, until they behave as humans do. Align: we test whether those agents compute as the brain does, and rebuild them wherever the two diverge. An agent aligned with people in both behavior and neural computation would be a working, mechanistic explanation of how the mind works.

Next: 1 Record

The research program as a cycle of three objectives around the idea of understanding by building neurally and behaviorally aligned agents grounded in human experience. Objective 1, Record: capturing human development in the real world. Behavioral alignment, modeling natural development, leads to Objective 2, Build: building AI agents that learn as humans do — multi-modal, RL, embodied, social. Neural alignment, brain-like architectures and learning rules, leads to Objective 3, Align: aligning artificial and biological neural computation. Expanding context — sensory modalities, ages, situations — leads back to Objective 1.

Select an objective in the figure to see how we pursue it.

Read the framework paper

Understanding by building agents that learn like the human mind and brain

Hasson U, Goldstein A, Raviv H. Trends in Cognitive Sciences.

PDF