Making Sense of the World (and Oneself)
From Five Senses Through Synesthesia and Beyond
prepared in association with an exhibit at the Painted Bride
Five senses | Seeing/hearing similarities | Hearing with eyes | Brain as perceiver | More than five senses | Perception as construction | The bottom line
A starting place: the "five senses"
You were probably taught in school that humans have five senses - seeing, hearing, touching, smelling, and tasting - each associated with a different part of the body - the eyes, the ears, the skin, the nose, the tongue. And that seems to make some sense. You probably experience seeing something as quite different from hearing it, or touching it, or smelling or tasting it. And you probably can do any one of those without the others. You can still hear something when you close your eyes, still see something when you cover your ears, and so forth.
Synesthesia = A condition in which one type of stimulation evokes the sensation of another
For some people, though, the idea of five independent senses makes less sense. Some people with synesthesia, for example, may experience sounds when they look at particular colors or colors when they hear particular sounds. How could that be? Are there really five independent senses? If so, people who experience sounds and colors together must be in some way "messed up." But maybe its actually the idea of five independent senses that is somehow not quite right, and synesthetics can help us better understand how we all actually make sense of the world? Let's explore that possibility ...
Sounds and sights aren't actually that different for any of us
Maybe its not so odd that people report seeing visual sensations when they hear things or having auditory sensations when they see them? Maybe that's something we all do, to varying degrees? . Try saying out loud to someone you know the words "bouba" and "kiki". And then ask them which of the two shapes to the right is named by each word. Or watch this video. Most people will say quite naturally that the "bouba" sound names the smoother, rounder shape and the "kiki" sound the sharper, more jagged one. Similarly, you'd be very surprised if a flash of lightening were accompanied by a soft, swooshing sound, or if a the ripples on a pond were associated with loud cracking noises. Softness isn't a sound or a sight; its either of them ... or both of them.
Maybe our brains aren't actually organized in terms of five senses at all, but according to some other set of principles that causes us to notice and pay attention to similarities (and differences) between what we hear/see/taste/smell/feel? Maybe the brain primarily interprets input from the eyes, from the ears, from wherever in relation to its own organization rather than in relation to where it comes from?
What we hear is influenced by our eyes as well as our ears
In everyday life, we don't normally see things separately from hearing them, or, often, from touching/smellling/tasting them. Our brain is normally getting input from our eyes, ears, skin, mouth, and nose all at the same time. So, is what we hear only due to our ears, or might other parts of our body be involved as well? It turns out that our perception of even very simple sounds are influenced by our eyes as well as our ears. The McGurk effect provides a simple illustration of this. Be sure to play this video first with your eyes closed and decide what you're hearing. Then play it while watching. Notice that what you hear is different when you have your eyes open. Clearly the "sound" is not determined by what your ears are doing but by your eyes as well.
We don't see things with our eyes, hear them with our ears, etc. Instead, we see see/hear/feel/smell/taste things with our brains
If we start with the idea that we learned in school, that our experience of the world depends on five independent senses each associated with a different part of the body, its a little strange to discover that what we hear is influenced by our eyes. But that begins to make some sense if we think about the brain. Information from the ears and the eyes comes together in the brain. If hearing happens in the brain, instead of in the ear, then its not so surprising that what we hear can be affected by what we see.
The neuroscientist Paul Bach y Rita took seriously the idea that "we don't see with our eyes, we see with our brains" and showed that blind people could reacquire some vision by using a videocamera that activated tactile inputs. Such "sensory substitution" is being explored and used in a variety of contexts, including providing some enhanced sense of sight using video cameras that produce sound signals. So maybe we don't in fact see with our eyes, but rather with our brains? And the same, of course, for our other senses?
That may seem a little hard to get one's head around, but think about dreaming. You see things when you're dreaming, don't you? But your eyes are closed, so what you're seeing isn't because of anything going on in your eyes. You probably hear things when your're dreaming too, and perhaps other times as well, even though there's nothing going on in your ears. Not only can one sense things using unusual parts of the body, but one can sense things just because of the brain, with no inputs from other parts of the body at all. What you see is clearly a construction of the brain, something that is influenced by your eyes but can be made independently of them. The same is true of hearing, feeling, smelling, and tasting. And each of these "senses" are themselves constructions of the brain, constructions that may be influenced by several different other parts of the body and may occur without them as well .
Are you standing upright or on our head? Are you standing still or moving? How do you know? Would you know if you couldn't see/hear/feel/taste/smell? In addition to input from your eyes, ears, skin, nose, and mouth, your brain gets input from a set of sensory receptors in a special part of your inner ear called the vesibular apparatus. These receptors contribute to "body sense," an awareness of position and movment in space, and you can indeed say whether you're upright or upside down and whether you are standing still or moving using them. But your body sense is strongly affected as well by other inputs, including inputs from your eyes. If you're looking out the window of a train and the train next to you starts moving, you will sometimes feel that you are moving, just because of the visual input.In addition to input from the vestibular apparatus, your brain is also constantly getting input from sensory endings embedded in your muscles and joints about their lengths and angles and the forces being experienced in various parts of your body. If you didn't have that proprioceptive information coming in, your brain couldn't keep track of all the parts of your body and how they relate to one another and you'd collapse. Your brain is also getting information about air born chemicals in your environment, "pheromones" that affect your behavior but don't produce a recognizable sensation.How come you didn't know about the brain's input from the vestibular apparatus or from the muscles and joints? Or about pheromones? How come you're taught in school about only five "senses," when in fact your brain is getting sensory information from many more than five other parts of your body? The "five senses" are those which most people are conscious of, those that most directly relate to different other parts of the body, and hence those that neuroscientists have studied most extensively. But you shouldn't take what you learn in school, or hear from scientists, too literally. Sometimes what you learn in school and/or what scientists tell you makes sense and you should use it when it does. Other times, it doesn't and you should question it. Stories are made to be revised, based on new experiences. So here's a new story based on new experiences: senses are made up in the brain, not created in the rest of the body, and you have more than five of them.
Good new scientific stories not only provide new answers to old questions but raise new questions as well, and the idea that sense are made up in the brain certainly does. You probably thought that your senses were your windows on the world, the portals through which you discover what is out there. If senses are made up by the brain, though, then maybe that's not true either? If your sense aren't there to discover what's out there in the world, what do you have them for?Let's look in a little more detail at your visual sense, and see if we can come up with a new scientific story of what your senses are doing, one that might help us understand better what senses are for in general.Optical illusions, like that shown to the right, are a good place to start. You see the square labelled A as dark and the square labelled B as light, but is that really what's out there? Actually not, the two squares are both sending exactly the same amount of light to your eye. If you don't believe it, click here.What you see isn't actually what's out there but instead an "informed guess." Your brain constructs a picture of what's out there based on what it thinks is important for you. In this case, your brain thinks its more important to notice coherent objects, including a checkerboard, than it is to construct a picture that shows exactly how much light is coming from each location. Its a guess that makes sense given that the actual light coming from different locations will vary a lot depending on the source of illumination. But its still only a guess, not what is actually out there.Its nice to know what you sense is an informed guess, but it may still be a little disconcerting to know its not actually what's out there. On the other hand, since the brain isn't trying to say what's actually out there (and may not in fact be capable of doing that) its free to try out different guesses and see which ones work best. Ambiguous figures, like that shown to the left, help to make this point clearly. The dotted corner of the cube may appear to you to be either sticking out toward you or sticking backwards. If you wait awhile though, you can probably see it the other way; your brain will try out a different guess. And once you get used to the idea that you can always see things in several different ways, you can use that to create things you've never seen before, like the image to the right below.
The brain constructs not only what you see but what you hear, feel, taste, and smell, in the same way: it makes informed guess about what's important and you can, if you want, ask it to try out a different guess. And it does the same thing with senses themselves. So you not only have more than five senses but can make more when you need or want them. That's useful when, for example, you're learning to drive car: your brain creates a new "sense of the road" to help you. And it can be fun, whether its particularly useful in any given case or not. Maybe that's part of the point of not only synesthesia but art in general? As you work your way around the exhibit on Synesthesia (and in life in general), see if you can find new ways to perceive things yourself.
- You're not limited to five senses; you have and can make more.
- Senses are constructed in the brain, using a variety of inputs ... and sometimes none at all
- Senses are a way not of knowing what's out there but of making informed guesses about it
- You can try out new ways of making sense of what's out there ... maybe that's what synesthetes are doing?
- You make the world and yourself by perceiving it, and you can remake both
- Enjoy the exhibit and your life in that spirit