Sensation and perception refers to the study of how we see, how we hear, how we smell, how we feel touch. The difference between sensation and perception is that in sensation we’re talking about the transduction of signals. So in the case of vision for example, sensation is really just focused on how light, that’s out in the world, is transduced, becomes a signal inside the brain. So it’s turned into an electrical signal. It’s no longer light. Perception is how we understand or interpret what those signals mean. So it’s about the implications, the meaning, that we associate with those signals. The importance of sensation and perception is in some ways obvious. I mean, we’re studying how it is that we see, how it is that we hear. These are critical abilities. These are the fundamental ways that humans and other animals get information about their environment in order to exploit their environments. 

The main challenge one experiences teaching sensation and perception is that so much of what you’re talking about are things that you want people to be able to hear or see, or feel, touch. So it’s the kind of area where just learning from a textbook, for example, or just reading papers often is inadequate or it sort of sidesteps the actual issue. So you can describe how something works and yet not really have a good feel for it if you don’t actually see it. And so one thing that these videos do is they re-create illusions or phenomenon for anybody to experience. Sometimes you can experience them directly in the videos that we’ve made or sometimes the videos show you how to create those experiences for a participant or for a class. But the bottom line is that when you teach sensation and perception and say, you talk about a visual illusion, for example, you want people to be able to see and experience that illusion. Just describing it is sort of insufficient for making it clear how it’s a phenomenon that requires an explanation. And so these videos supply that experience, that visual or auditory experience, or the methods for producing a tactile experience that’s so important for the study of sensation and perception. 

Illusions have for a very, very long time played a role in the study of sensation and perception. I think that there’s really two reasons for that. One reason is that illusions reveal something about how the human brain or any organism’s brain really solves a problem that has no answer. And so figuring out what’s out there in the world on the basis of the given input requires a leap, it requires an extra step, it requires some inference above and beyond just the input itself. And so whenever such inferences are made, it’s based on assumptions. It’s based on heuristics. It’s based on tricks, but fundamentally it’s a system that doesn’t actually know the answer. It doesn’t actually, it’s not 100% certain that it knows the truth about the world. So illusions show that. They’re sort of tricks that take advantage, that exploit the assumptions and the heuristics that the brain is using to make inferences about the world. So the illusions are very strong tests of a theory or a hypothesis about how the brain is solving some problem. A great example of the role that illusions play in the study of perception is, for example, the Ames room in this video series, which is an illusion that tells us something about how the brain perceives 3-D space, despite the fact that the inputs that it receives are only two-dimensional. So in order to make inferences about how far away objects are in space and also how big objects are, the brain needs to use tricks or have assumptions or use heuristics, because the input on the retina are only two-dimensional. So objects that are far away will project small images on the retina. And likewise small objects that are nearby will project small images on the retina. So how’s the brain supposed to figure out if an image on the retina is small because it’s from an object that’s far away or an object that’s actually small. There’s no mathematical way to solve that problem with 100% certainty. So the brain makes assumptions. The Ames room is an illusion that takes advantage of those assumptions. In particular, the assumption there is that angles tend to be right angles. But in any case, what the Ames room shows is that the brain’s understanding of distance and size is completely confounded, and the brain is sort of making guesses or inferences about the true relationships between objects in the world. 

The reason why research turned in the direction of sensation and perception the 19th century is because of physics, because there were many researchers who started to understand the properties of light and were interested in optics. They were interested in, they were making glasses and grinding lenses, and figuring things out about how light works and how to bend light and also how to bend it for human purposes. And in doing so they started to ask questions about how the human visual system actually works. They discovered many, many of the most basic principles that we still rely on today for understanding sensation and perception and especially human vision. One of the reasons that the research they did is so cool and important back then is because they did it without any access to the human brain really. So they weren’t recording from neurons. They didn’t have scanners for imaging human brains and looking at brain waves, and they didn’t really have many fancy tools at all. Yet, the hypotheses that they generated and the things that they learned about sensation and perception ended up proving out to be true, not just in a theoretical or psychological sense, but also in a neural sense. We only found that sort of evidence 100 or 200 years later. So in some ways I think of sensation and perception as one of the best examples of the scientific method at work. It was only later in the 1960s, ’70s and ’80s that we started to discover via neurophysiology that there really are cells that literally are sensitive to the presence of some colors and the absence of their opponent colors. So in many ways that work from the 19th century made a prediction about the existence of certain subtypes of cells in the human brain, of neurons in the human brain. It was only 100, 200 years later that the existence of those cells was confirmed by modern recording techniques in the brain. 

I think there are really two key questions that will start to address or that we’re starting to address and that we’ll start to get a better grip on in the next century or so in the area of sensation and perception. The first question is about how does the human brain produce conscious experience. So we can think about sensation and perception in terms as a mathematical problem. Here’s some input and can you do some math on that input to figure out where it came from? In other words, what the world that produced that input must be like. But nowhere in there do you have to have an experience of the input or an experience of the world that came from. So we can write in fact because we know so much about how the human brain does these things, we now write computer programs that can solve very sophisticated problems, very sophisticated visual problems. So we can write a computer program that recognizes a face, for example. But the computer program doesn’t have a feeling like it’s looking at a face. There’s no sense in which it has like a mind’s eye and the face it’s looking at is in that frame. But we all experience that. You’re watching the video right now and you have an experience of me sitting there and where is that experience? I know where I am in the video. I mean, I’m here right now and there’s a camera pointed at me and it’s recording. Someone, you out there, you’re watching it at some other time. Where’s your experience of the video? It lives somewhere in your brain and we really don’t understand that. Scientists are going about that. So a good example is the rubber hand illusion. The rubber hand illusion is a very strange illusion where people start to feel like a rubber hand is their own, and in fact they think that they feel sensation in that hand, in that piece of rubber, that is not a part of their body. So that strange illusion reminds us that even the feeling of touch, sort of the feeling itself, the experience of touch is an invention of the brain. It can be misattributed. And those kinds of illusions of conscious perception should start to help us to understand how the human brain produces the experience of conscious perception and why. Another set of issues that’s at the forefront of research in sensation and perception these days I think has to do with language and how language interfaces with sensory and perceptual systems. At the end of the day, one thing that we do with perception is talk about it. We talk about what we see. We talk about how things taste. We talk about what we feel physically. So how do we do that? How do words map on to the outputs of sensory and perceptual processes?