Minds as simulators
The simulations that brains run: dreaming, imagining, modelling other minds, play, and building models of the body and the world.
Many accounts in psychology and neuroscience say brains don't just register the world. They simulate it. Dreaming replays and recombines the day's experience, and none of it is real. Rehearsing a movement before making it, working out what another person thinks, and a child feeding a toy dinner all do the same thing. Each runs a model forward and reads off the result before acting, or instead of acting. Predictive accounts go further. They describe perception, memory and social thinking as simulation: always predicting what comes next and correcting for what arrives. Enactive and ecological accounts reject that picture of perception. This section covers the simulations documented in humans and other animals, not the computers built to imitate them.
REM dreaming
The vivid, narrative dreams of rapid-eye-movement sleep, built from memory fragments and emotion with little conscious control.
Question it answers: What does the sleeping brain construct once it is cut off from the senses?
Leaves out: Physical consistency: flying, dead relatives and rooms that rearrange themselves pass without complaint.
Runs on: about 1016 FLOPS wild guess a sleeping human brain; Sandberg and Bostrom-style estimateDocumented since: 1953 Aserinsky and Kleitman's discovery of REM sleepIn the simulations tree: Dreams
Lucid dreaming
Knowing you are dreaming while the dream continues, sometimes with enough control left over to steer what happens next.
Question it answers: Can a dreamer recognise, from inside it, that it is a dream?
Leaves out: Awakening, which commonly interrupts the episode just as it becomes absorbing.
Runs on: about 1016 FLOPS wild guess a sleeping human brain; Sandberg and Bostrom-style estimateDocumented since: 1913 Frederik van Eeden coined the term from his dream diaries
Hippocampal replay in rats
Place-cell firing sequences recorded while a rat runs a maze, then replayed in the same order during rest and sleep.
Question it answers: Does the brain rerun a route it has already travelled?
Leaves out: Anything the animal has not just experienced: replay only reruns the recent past.
Runs on: about 1016 FLOPS wild guess a rat brain; Sandberg and Bostrom-style estimateDocumented since: 1994 Wilson and McNaughton's report of hippocampal reactivation during sleepIn the simulations tree: Hippocampal replay
Threat-simulation theory of dreams
The idea that dreams exist to rehearse dangerous encounters, tested by counting threat content across many recorded dreams.
Question it answers: Do dreams train the brain to handle threats it has not yet met?
Leaves out: Explaining why so many dreams concern mundane failures, such as missing a bus, rather than threats.
Runs on: about 1016 FLOPS wild guess a sleeping human brain; Sandberg and Bostrom-style estimateDocumented since: 2000 Antti Revonsuo's threat-simulation theory of dreaming
Sleep-dependent memory consolidation
Skills and facts learned before sleep, tested again afterwards to see what the brain reprocessed and strengthened overnight.
Question it answers: What does the brain rehearse and strengthen while the body is unconscious?
Leaves out: It doesn't say which memories get picked, or why those and not others.
Runs on: about 1016 FLOPS wild guess a sleeping human brain; Sandberg and Bostrom-style estimateDocumented since: 1924 Jenkins and Dallenbach's nonsense-syllable retention study
Hypnagogic imagery
Vivid, fragmentary perceptual images that appear as the brain disengages from the senses while falling asleep.
Question it answers: What does the brain generate when input from the world is fading but consciousness has not yet switched off?
Leaves out: Coherence: hypnagogic images rarely add up to a story.
Runs on: about 1016 FLOPS wild guess a drowsy human brain; Sandberg and Bostrom-style estimateDocumented since: 1848 Alfred Maury's description of hypnagogic hallucinations
Hippocampal preplay
Place cells fire in sequences for places never yet visited, as if the brain rehearsed paths in advance.
Question it answers: Does the brain simulate journeys before it takes them?
Leaves out: Later work questions how much of preplay is truly predictive.
Runs on: no computer. the brain, no computation countedDocumented since: 2011 by 2011, Dragoi and Tonegawa, Preplay of future place cell sequences source
Mental rehearsal in athletes
Athletes running through a routine or shot in their head, activating much of the same motor circuitry as performing it for real.
Question it answers: Can practising a movement only in imagination improve performance of the real thing?
Leaves out: Fatigue, anxiety and the genuine risk of failure.
Runs on: about 1016 FLOPS wild guess an athlete's brain; Sandberg and Bostrom-style estimateDocumented since: 1932 Edmund Jacobson recorded muscle activity during imagined movement
Episodic future thinking
Constructing a detailed personal scene of a specific future event, using much of the same machinery used to recall the past.
Question it answers: What will a particular future moment be like, in enough detail to plan for it?
Leaves out: Accuracy: imagined futures borrow freely from memory and are often wrong in the details.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 2001 Atance and O'Neill named episodic future thinking
Counterfactual thinking
Replaying a past event with one detail changed, to see how differently things would have turned out.
Question it answers: What would have happened had the agent acted otherwise?
Leaves out: Ever running the real alternative to check the answer.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 1982 Kahneman and Tversky's simulation heuristic paper
Mental rotation
Rotating an imagined three-dimensional object in the mind's eye to decide whether it matches another view of the same shape.
Question it answers: Is this the same object, seen from a different angle?
Leaves out: The object entirely: nothing physical actually turns.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 1971 Shepard and Metzler's mental rotation experiment
Mind-wandering and daydreaming
Spontaneous, self-generated trains of thought about hypothetical people, places and events, running whenever attention is not needed elsewhere.
Question it answers: What does the brain simulate when it has nothing else to do?
Leaves out: Staying on topic.
Runs on: about 1016 FLOPS wild guess an idle human brain; Sandberg and Bostrom-style estimateDocumented since: 1966 Singer's systematic study of daydreaming
Constructive episodic simulation
Remembering is a rebuilding of past events from stored pieces, the same system that imagines what might happen.
Question it answers: Is recalling the past the same act as imagining the future?
Leaves out: It is a hypothesis about memory, not a mechanism with parameters.
Runs on: no computer. the brain, no computation countedDocumented since: 2007 Schacter and Addis’s account of remembering and imagining as one constructive system sourceIn the simulations tree: Imagining the future
Motor imagery
Imagining a movement without doing it, which engages much of the same neural machinery as performing it.
Question it answers: Does imagining a movement run the brain's motor program?
Leaves out: Imagery can be vague or absent, and its timing varies among people.
Runs on: no computer. the brain, no computation countedDocumented since: 1994 by 1994, Jeannerod, The representing brain: neural correlates of motor intention and imagery source
Visual mental imagery
Pictures in the mind's eye that can be scanned, zoomed and inspected, with effort rising with distance.
Question it answers: Is a mental image like a picture that can be scanned?
Leaves out: Whether images are pictorial or descriptive remains disputed.
Runs on: no computer. the brain, no computation countedDocumented since: 1973 by 1973, Kosslyn, Scanning visual images: some structural implications source
Affective forecasting
People simulate how they will feel in a future event and mispredict its intensity and duration.
Question it answers: How well can people predict their own future feelings?
Leaves out: The forecast is of a few feelings, not of a full life.
Runs on: no computer. the brain, no computation countedDocumented since: 1998 by 1998, Gilbert and others, Immune neglect source
Mental models of reasoning
The mind holds a small-scale model of how things work and uses it to try out actions before taking them.
Question it answers: Does reasoning work by running a small model of the world?
Leaves out: Craik gave an idea, not a testable theory of how models are built.
Runs on: no computer. the brain, no computation countedDocumented since: 1943 by 1943, Craik, The Nature of Explanation sourceIn the simulations tree: Craik’s model in the head
Vicarious trial and error
A rat pauses at a choice point and looks back and forth between options as if trying each in turn.
Question it answers: Does a rat deliberate by sampling the outcomes of paths?
Leaves out: Behaviour shows deliberation, but not what is in the animal's mind.
Runs on: no computer. the brain, no computation countedDocumented since: 1938 by 1938, Tolman, The determiners of behavior at a choice point source
Future planning in animals
A bird caches food in the place where it expects to be hungry, acting on a simulated future need.
Question it answers: Can an animal plan for a need it does not feel now?
Leaves out: Simpler learning rules can explain some of these results.
Runs on: no computer. the brain, no computation countedDocumented since: 2007 by 2007, Raby and others, Planning for the future by western scrub-jays source
Theory of mind
Modelling what another person knows, wants or believes, including cases where that differs from what is actually true.
Question it answers: What does this other person think is going on?
Leaves out: Ever checking the model directly against what they actually think.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 1978 Premack and Woodruff coined theory of mind
Empathy as simulation theory
Feeling something of another person's state by running a version of their situation through one's own emotional and bodily systems.
Question it answers: What would the observer feel in the other person's position?
Leaves out: The difference between self and other, which is where empathy most often fails.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 1986 Gordon's simulation theory of folk psychologyIn the simulations tree: Simulating other minds
Mirror systems
Neurons that fire both when an action is performed and when it is watched in someone else.
Question it answers: Can understanding another's action be simulating it in one's own motor system?
Leaves out: Explaining understanding that has no matching action, such as watching a fish swim.
Runs on: about 1016 FLOPS wild guess a macaque brain; Sandberg and Bostrom-style estimateDocumented since: 1992 Di Pellegrino, Fadiga and colleagues report mirror neurons
Audience design
Predicting what a listener already knows and needs explained, before choosing how to say something.
Question it answers: What must the speaker simulate on the listener's behalf?
Leaves out: Directly querying the listener.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 1987 Isaacs and Clark on references between experts and novices
Visual perspective-taking
Working out what another person can see from where they are standing, by mentally moving to their point of view.
Question it answers: What does the scene look like from the other person's position rather than the observer's own?
Leaves out: The step in toddlers, who assume that everyone sees what they see.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 1948 Piaget and Inhelder's three mountains task
Joint attention
Following another person's gaze or point to work out what they are looking at, then checking back to confirm the target is shared.
Question it answers: What is currently capturing their attention?
Leaves out: The underlying thought, tracking only the target of it.
Runs on: about 1016 FLOPS wild guess an infant's brain; Sandberg and Bostrom-style estimateDocumented since: 1975 Scaife and Bruner on joint visual attention in infants
Fiction as social simulation
Reading stories runs a model of social life, strengthening the reader's grasp of other people's minds.
Question it answers: Does fiction train the mind to simulate other people?
Leaves out: Evidence for a lasting gain in social skill is mixed.
Runs on: no computer. the brain, no computation countedDocumented since: 2008 by 2008, Mar and Oatley, The function of fiction is the abstraction and simulation of social experience source
Children's pretend play
A child treating a banana as a telephone, or an empty cup as full of tea, running an invented layer of meaning over the real objects.
Question it answers: What would happen if this object were something else?
Leaves out: Any objection when the pretend layer contradicts the real one.
Runs on: about 1016 FLOPS wild guess a child's brain; Sandberg and Bostrom-style estimateDocumented since: 1945 Piaget's study of symbolic play in childrenIn the simulations tree: Children’s pretend play
Animal play as practice
Young mammals wrestling, chasing and stalking without consequence, using movements drawn from real hunting and fighting.
Question it answers: Does harmless play rehearse skills the animal will need for real later?
Leaves out: Establishing whether play is merely intrinsically rewarding.
Runs on: about 1016 FLOPS wild guess a young mammal's brain; Sandberg and Bostrom-style estimateDocumented since: 1898 Groos's practice theory of animal play
Imaginary companions
A sustained, named, invented character that a child talks to and negotiates with over weeks or years.
Question it answers: Can a fully invented other mind still be useful to think with?
Leaves out: Continuity: for most children the companion fades by about age seven.
Runs on: about 1016 FLOPS wild guess a child's brain; Sandberg and Bostrom-style estimateDocumented since: 1934 Svendsen's early study of children's imaginary companions
Sociodramatic play
Groups of children jointly acting out invented roles and scenes, negotiating the rules of the pretend world as they go.
Question it answers: Can several separate imaginations run the same simulated scene together?
Leaves out: Sustained maintenance of character.
Runs on: about 1016 FLOPS wild guess several children's brains; Sandberg and Bostrom-style estimateDocumented since: 1933 Vygotsky's lecture on play and child development
Paracosms
An elaborate invented world, with its own geography, history and rules, built and maintained in the imagination over months or years.
Question it answers: How much internal consistency can an invented world sustain?
Leaves out: Any requirement that others share the belief.
Runs on: about 1016 FLOPS wild guess a child's or adult's brain; Sandberg and Bostrom-style estimateDocumented since: 1826 the Brontë children’s Glass Town, from 1826; Silvey’s survey of childhood paracosms followed in 1976 sourceIn the simulations tree: Glass Town
Object play in primates
Wild chimpanzees carrying and cradling a stick the way they would hold an infant, well past the age of ordinary play.
Question it answers: Can play stand in for a skill an animal has not yet had a chance to practise?
Leaves out: Knowing whether the chimpanzee is pretending in any human sense.
Runs on: about 1016 FLOPS wild guess a chimpanzee's brain; Sandberg and Bostrom-style estimateDocumented since: 2010 Kahlenberg and Wrangham on stick-carrying in wild chimpanzees
Predictive processing
The brain generating a running prediction of its own sensory input, and updating only on the mismatch with what arrives.
Question it answers: What does the world probably look like right now, before the senses even report it?
Leaves out: Explaining where the very first prediction ever came from.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 1999 Rao and Ballard's predictive coding model of visual cortex
Motor forward models
A copy of a movement command run forward to predict its sensory consequences, ready before the real feedback arrives.
Question it answers: What will this movement feel and look like, before it has finished?
Leaves out: The actual movement, which occasionally disagrees with the prediction.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 1995 Wolpert, Ghahramani and Jordan's internal-model paper
The body schema
A running model of where the body's own parts are and how large they are, updated by touch, sight and movement, and susceptible to manipulation, as in the rubber-hand illusion.
Question it answers: Where are the body's parts, and where does the body end?
Leaves out: Frequent obvious error, which makes illusions striking when it does occur.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 1911 Head and Holmes introduced the body schema
Hallucination as uncontrolled prediction
Perception with the balance tipped too far towards prediction and away from incoming evidence, so the model is experienced as though it were the world.
Question it answers: What happens to perception when prediction stops being corrected by evidence?
Leaves out: Explaining every kind of hallucination equally well.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 2009 Fletcher and Frith's Bayesian account of hallucination
Blind-spot filling-in
The visual system generating plausible content for the small blind spot in each eye's field of view, matched to the surrounding pattern.
Question it answers: What is probably there, in the one place the eye cannot see?
Leaves out: Any verification of the inferred content, since nothing appears obviously missing.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 1992 Ramachandran's studies of blind-spot filling-in
Interoception
A running prediction of the body's internal state, heartbeat, breathing, hunger and gut, checked against the actual signals arriving from inside.
Question it answers: What is the current internal state of the body?
Leaves out: Conscious access: most people can report on it only rarely.
Runs on: about 1016 FLOPS wild guess a human brain; Sandberg and Bostrom-style estimateDocumented since: 2013 Seth's predictive account of interoceptive inference
Thought as simulated action
Thinking is covert action and perception: the brain runs its own motor and sensory systems offline to rehearse outcomes.
Question it answers: Is conscious thought the brain simulating its own behaviour and perception?
Leaves out: Competing accounts explain the same results with symbols.
Runs on: no computer. the brain, no computation countedDocumented since: 2002 by 2002, Hesslow, Conscious thought as simulation of behaviour and perception source
Grounded cognition
Concepts are reactivations of sensory and motor states stored from experience, not abstract symbols.
Question it answers: Does understanding a concept re-run the experience of it?
Leaves out: Abstract concepts are hard to explain by sensory reenactment.
Runs on: no computer. the brain, no computation countedDocumented since: 1999 by 1999, Barsalou, Perceptual symbol systems source
The phenomenal self-model
The sense of being a self is a model the brain builds of its own body and state and cannot see as a model.
Question it answers: Is the self a simulation that the brain does not recognise as one?
Leaves out: It is a philosophical framework, not an experimentally tested mechanism.
Runs on: no computer. the brain, no computation countedDocumented since: 2003 by 2003, Metzinger, Being No One source
Body ownership illusions
Brushing a hidden hand and a visible rubber hand together makes the rubber hand feel like one's own.
Question it answers: How does the brain decide which body is yours?
Leaves out: Strength of the illusion varies widely among people.
Runs on: no computer. the brain, no computation countedDocumented since: 1998 by 1998, Botvinick and Cohen, the rubber hand illusion source
Embodied language simulation
Understanding a sentence involves mentally simulating the scene it describes, including shapes and orientations.
Question it answers: Do readers build a mental scene while reading?
Leaves out: Some replications give weak effects.
Runs on: no computer. the brain, no computation countedDocumented since: 2001 by 2001, Stanfield and Zwaan, implied orientation and picture recognition source
The intuitive physics engine
The brain predicts how objects will fall, collide and stack by running approximate physical simulations in the head.
Question it answers: Does judging a falling tower use an internal physics simulation?
Leaves out: The model fits some judgements, and fails on others.
Runs on: about 109 FLOPS rough a laptop, seconds per judgementDocumented since: 2013 by 2013, Battaglia, Hamrick and Tenenbaum, Simulation as an engine of physical scene understanding source