Stop, children what's that sound? Multi-modal inference through mental simulation

Abstract

Human adults can figure out what happened by combining evidence from different sensory modalities, such as vision and sound. How does the ability to integrate multi-modal information develop in early childhood? Inspired by prior computational work and behavioral studies with adults, we examined 3- to 8-year-old children’s ability to reason about the physical trajectory of a ball that was dropped into an occluded Plinko box. Children had to infer in which one of three holes the ball was dropped based on visual information (i.e., where the ball landed) and auditory information (i.e., the sounds of the ball colliding with parts of the box). We compare children’s responses to the predictions of four computational models. The results suggest that although even the youngest children make systematic judgments rather than randomly guessing, children’s ability to integrate visual and auditory evidence continues to develop into late childhood.