PREVNEXTSUBMITTable of ContentsResourcesGlossaryTranscriptClear and return to MenuSearch ResultsFilterSlide TextTranscriptSearch in:Check to includeSearch...TermsDefinitionFINISHfinishdefinitionresourcessearchpauseplayreplaysubmitnextpreviousvolumeSkip Navigation. Hit enter to return to the slide.SUBMIT ALLsubmit allQuestion ListMore infoSend an emailCloseBioExit<html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>Speech perception is an incredibly complicated process. It might not seem so, since even little kids can do it pretty easily and automatically, but experiments in speech research have shown that there is a lot going on, much of it that we still don’t really understand. This presentation will outline the general approaches that are taken in theories of speech perception and provide some highlights of experimental findings.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>Speech perception is necessarily complicated in part because humans are complicated. Each of us has a unique anatomy and so none of us produce speech in exactly the same way. In addition, speech itself has a variety of layers, from the fundamental frequency of phonation, to the resonances of the vocal tract, to the prosodic structures that the speech sounds appear in. Finally, all of this has to be learned, at least to some extent, by infants who grow up in different parts of the world and who learn different languages.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>There are many differences between speakers of the same language. My typical fundamental frequency, use of vocal fry, and the size of my vocal tract as a resonating cavity are all particular to me. Yet you fairly easily and rapidly were able to understand me from the very first words you heard in the very first video you listened to. You may have listened to interviews with seven-foot tall basketball players or had a conversation with a child. Despite these vast differences in vocal tract between speakers, we are all pretty good at identifying a vowel as /a/ or a fricative as /s/. Any theory of speech perception will have to explain how we can understand speech despite all of this variability between individuals.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>Speech is also a pretty complicated system, even within an individual speaker. For example, the fundamental frequency is raised on a stressed syllable. But the fundamental frequency is also raised for high vowels, though not as much. The point here is that the same piece of acoustic information is the result of a combination of several factors and to understand speech they all have to be sorted out. As another example, coarticulation can result in different resonance frequencies for “s”. When followed by a rounded vowel, the “s” is rounded too, and this will lower the resonance frequency for the noise in the “s”. There is another fricative, the /ʃ/, that has a lower noise frequency than “s”. So a listener will have to decide whether they are hearing a rounded “s” or a /ʃ/.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>A third factor that influences theories of speech perception is that speech perception must be learned in some way. Infants have to be able to learn whatever language it is that is being spoken around them, and different languages have different inventories of phonemes and allophones. A famous example of this is Japanese versus English. English has two distinct phonemes “l” and “r” where Japanese has just one phoneme that varies from somewhat more like an “l” to more like an “r” depending on context. Japanese adults who try to learn English have a very hard time learning this distinction. Babies born in English speaking countries have no trouble with this at all. So at some point when they were babies, Japanese speakers were able to hear this difference, too, but they then learned to categorize these variants as the same speech sound.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>Categorizing speech that you hear into the speech sound categories of your language is an important skill. For speakers of a language, this becomes automatic and in fact, speakers lose a lot of their ability to hear small differences between speech sounds that are in the same category. </textformat><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>We have perceptual categories in places other than speech that can provide a helpful analogy. For example, a spectrum of light frequencies changes from one color to the next, but if you look within a particular color (for me, the red and the green in this image) it looks the same over a wide band. The light frequency is changing continuously across the image but our brain categorizes it into discrete chunks with somewhat more abrupt looking changes at the edges.</textformat><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>You can play an example of some computer synthesized speech where the sound is changed acoustically by the exact same amount from one syllable to the next. When you listen to it though, it will sound like the same syllable over and over until it abruptly changes to something different. This continuum of synthesized speech goes from /ba/ to /da/ to /ɡa/. It turns out that this categorical perception is more robust for consonants than vowels. For vowels, people are better at hearing small differences, in other words, their perception is more continuous.</textformat><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>Image Credit</textformat><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>https://commons.wikimedia.org/wiki/File:Colours_of_the_visible_light_spectrum.png</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>Speech is an interesting perceptual category because, unlike something like light, we can both produce speech and perceive speech. While it would be really cool to be able to shoot light rays out of our eyes at different frequencies, we can’t do that. However, we can produce lots of different speech sounds. Because of this, people who make theories about how speech works have considered whether speech perception is really grounded in the sounds of speech, the speech acoustics, or whether speech production might play a special role.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>The Motor Theory of speech perception takes the fact that we are producers of speech as a crucial aspect of speech perception. The Motor Theory claims that we identify phonemes through access to the underlying targeted gestures that produced them, not directly through the acoustics of the sounds. So, in the Motor Theory, there is a special part of speech perception that requires speech production.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>There is some evidence in favor of the Motor Theory for speech. The fact that categorical perception occurs for consonants and not so much for vowels reflects the articulatory possibilities pretty well. You can position your tongue just about anywhere in the oral cavity and produce different formant frequencies, and therefore, different vowels. But when you make a closure or a narrow constriction there are some important landmarks, like the front teeth that have a dramatic impact on the acoustics. Any fricative produced near the front teeth will create a jet of air that hits the teeth, resulting in high frequency noise. Recall also the perturbation theory for formants. There are distinct articulatory regions that will bend the formant frequencies up or down. Distinct places of articulation are created that result in a regular pattern of formant frequency transition, and some variation around these places may change the amount of transition a bit, but will not change the up or down direction of the formant frequency change.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>A second piece of evidence in favor of the motor theory is that a lot of what we perceive goes back to the source and is not focused on the acoustics. This is known as direct perception. For example, you have probably been in a restaurant and heard the sound of a plate being dropped and breaking. In that case, the percept you have is not a jumble of frequencies, it is the cause of that noise. You can probably imagine the sound of a car door slamming, dropping your keys, the sound of a glass being filled with water. I could go on and on. In all of these cases what pops into your brain is not the sound itself, but the source of the sound.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>Another piece of evidence for the importance of articulation in speech perception comes from the so-called McGurk effect, named for the researcher who discovered it. The McGurk effect shows that visual information about articulation can affect how speech is perceived. You can experience the McGurk effect for yourself right now. I have an audio recording and also a video. First, listen to the audio and decide what it is that you hear. Then, watch the video and decide what it is you hear. Finally, click on the video again, but close your eyes and don’t watch. What you hear should change depending on whether or not you are watching. And in fact, the audio link just plays the audio from the video. It’s the same sound. One of the really fascinating things about the McGurk effect is that it works on you even if you know exactly what is going on. You can’t help but use what you can see in the video to decide what you are hearing, even after you know it is misleading.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>The other general approach to speech perception is probably the more obvious theory of how we perceive speech, that speech is perceived based on the acoustics. Given what I’ve said so far about the motor theory, this might seem less obvious now than it was a few minutes ago. But there is also some evidence that speech is perceived based on acoustics and not as a result of having special knowledge about how speech is produced.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>One strong argument against the Motor Theory for speech perception is that researchers have demonstrated categorical perception in experiments with animals listening to speech stimuli. One of the first experiments of this kind used chinchillas who were trained to move to one side of a box or the other depending on the category of the speech sound. The chinchillas showed categorical perception just like people did. Their behavior changed abruptly at a boundary rather than being continuous or more variable. Similar experiments have been done with Japanese quail and with other animals.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>Another argument for the acoustic nature of speech perception comes from properties of speech that are the same for everyone, though not as straightforwardly. As an example, when we were studying vowels there were generalizations about high versus low vowels. High vowels have a low F1 and low vowels have a high F1. At the time, I provided some typical frequencies for F1. Those specific frequencies will differ from person to person, but for every person, the F1 for high vowels will be lower than the F1 for low vowels. So, there is a pattern there that doesn’t depend on a specific number for the formant frequency. The same thing could be said for the formant transitions. The pattern of raising or lowering formant transitions may vary in how much the formant changes by, but whether it goes up or down for a particular place of articulation will be the same for everyone. So at a slightly more abstract level, there is a stable acoustic pattern in speech.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>The idea that a pattern of acoustic information can be enough to perceive speech can be demonstrated using what is called “sine wave speech”. Sine wave speech is synthesized speech using just three simple periodic waves. These simple periodic waves are synthesized to be the same as the first three formants. So much of the rich information in the speech signal, for example the fundamental frequency and the harmonics, is lost. But the time varying pattern of formant resonances is still represented by the pure tones. Understanding sine wave speech is difficult, but it only takes a little bit of information to be able to hear it clearly as speech. In the example here, first listen to the sine wave sentence. Then, click to reveal four possibilities for the sentence. I will also tell you that this particular example was created in England, so the speaker that was modeled has a British accent. With these bits of extra information, the sentence suddenly becomes very easy to recognize. And once you’ve heard it as a sentence, you can’t really hear it as just whistles any more.</textformat></html><html><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'>Given that there is evidence for both articulatory and acoustic information affecting how we perceive speech, it’s likely that the true theory of speech perception will involve aspects of both the motor theory and the auditory theory. It makes sense that we would use our knowledge about how speech sounds are produced, or the source of speech sounds, in understanding and perceiving speech. However, people are generally pretty bad at lip reading and find it very difficult. Ultimately, speech must be heard to be understood easily and automatically so the acoustic information appears to play a primary role.</textformat><textformat leading='0' tabstops='[0, 48, 96, 144]' blockindent='0' indent='0'></textformat></html>