Selective attention for audiovisual integration of speech
Loading...
Date
2022
Authors
Boutros, Sylvain John
University of Lethbridge. Faculty of Arts and Science
Journal Title
Journal ISSN
Volume Title
Publisher
Lethbridge, Alta. : University of Lethbridge, Dept. of Neuroscience
Abstract
The perceptual brain decomposes the audiovisual world into a set of audiovisual features
such as color, shape, and pitch. An important question in perception science is whether
selective attention is required to bind audiovisual features back into unified perceptual objects.
In visual displays, targets defined as conjunctions of bound features typically cannot
be searched for in parallel across a complex scene. Instead, attention must be scanned
through a visual scene to find such targets. This means that conjunction of features requires
selective attention. Only a few prior studies have investigated this process in the
crossmodal audiovisual case. These prior studies left some ambiguity as to how temporal
dynamics interact with selective attention, and none have used dynamic audiovisual speech
as stimuli. In two experiments, this thesis explored whether the crossmodal conjunction of
audio and visual speech features requires selective attention. In Chapter 2, we presented
observers with displays of multiple visual faces and one audible voice. The task was to
determine whether one of the faces was saying a sentence that matched the voice. In Chapter
3, we presented observers with displays of multiple audible voices and one visual face.
Similarly, the task was to determine whether one of the voices was saying the sentence that
matched the lip movements of the visual face. We compared response times to complete
the task as the number of distracting items was increased. We found that it took longer to
complete tasks with increasing number of distractors. This means that audiovisual speech
targets cannot be registered in parallel across the scene and strongly suggests that selective
attention is required to bind speech features across modalities.
Description
Keywords
neuroscience , audiovisual , psychology , cognitive science , auditory , visual , visual perception , auditory perception , speech integration , Neurosciences , Psychology , Cognitive neuroscience , Auditory perception , Visual perception , Speech perception , Selectivity (Psychology) , Human information processing , Attention , Distraction (Psychology) , Dissertations , Academic