How does the brain make sense of its senses? A new book co-authored by Daniel Goldreich an associate professor in the department of Psychology, Neuroscience and Behaviour, and his colleagues Wei Ji Ma at New York University, and Konrad Kording from the University of Pennsylvania, explores human perception.

Whenever we look, listen, smell, taste, or touch, we have the impression that we perceive effortlessly. But in reality, perception emerges only after the brain’s neural circuits undertake a sophisticated weighing of probabilities — because any sensory input is open to interpretation.

Consider the image above. The brain’s goal is not merely to see this image, but to interpret it.

In essence, the brain asks itself: What in the world caused this particular arrangement of shapes and colours that I’m seeing? Were inks of various colours simply splattered haphazardly onto the page?

While a random splattering could have produced the image, the specific shapes and colours suggest that this origin story is implausible.

A more likely explanation is that the green shapes, the central cement-coloured strip, and the blue background represent a three-dimensional scene: a tree-lined sidewalk on a sunny day.

The converging lines suggest that the sidewalk is extending into the distance. All of this unconscious reasoning, based on probabilities called likelihoods, allows the brain to make an educated guess about what it’s seeing.

Importantly, the brain’s perceptual conclusions are based not only on the sensory input but on the context of the situation. If you were standing outside when you experienced this scene, your prior probability, given the outdoor context, would strongly favour a 3D interpretation.

By incorporating both likelihoods and prior probabilities into its reasoning, the brain can achieve an optimal interpretation of the scene, a Bayesian perceptual inference.

The same scene provides the opportunity for further inference. What about that person walking towards you? Who is it? Here, the brain weighs additional likelihoods and prior probabilities. For instance, if the shape of the distant person seems to resemble that of your friend (a high likelihood) and you know that your friend often walks in the area (a high prior probability), you would perceive the individual to be your friend.

The new textbook, Bayesian Models of Perception and Action: An Introduction (MIT Press, 2023), is a first-of-its-kind introduction to the field of Bayesian perceptual modelling, providing step-by-step procedures for constructing optimal probabilistic inference models that can be compared to human performance on perceptual tasks.