Chapter 5: Higher-Level Visual Processing: Beyond V1

Introduction

In our exploration of sensation and perception, we've traced the visual processing pathway from the eye to V1 (striate cortex) and discussed the divergence of information into the dorsal and ventral pathways. Although these pathways interact extensively, they become increasingly specialized as visual information progresses through the cortex, allowing different aspects of vision to be processed in parallel. These pathways are crucial for higher-level visual processing, and they were first extensively studied by neuroscientists Leslie Ungerleider and Mortimer Mishkin. In this chapter, we'll delve deeper into these pathways, their neural basis, and their functions.

Dorsal and Ventral Pathways: A Fishy Mnemonic

The "Two-Stream" hypothesis makes a distinction between "What" and "Where/How" processing (Ungerleider & Mishkin, 1982; Goodale & Milner, 1992). For a memorable mnemonic, think of a fish: if you were a fish and had a

dorsal fin on your back and continued this pathway from your back to the back of your head and over the top of your head then you would be tracing the dorsal stream

("Where" or "How" pathway) which travels to the parietal lobe, whereas the ventral stream ("What" pathway) travels from the back of your head down to the temporal lobe.

Dorsal Pathway (The "How" Stream)

• The dorsal pathway extends from V1 to the parietal lobe.
• Often referred to as the "Where" or "How" stream, it primarily deals with spatial information and guiding actions in response to visual input. Originally the dorsal stream was described primarily as a "where" pathway because of its role in spatial perception. More recent research has emphasized its importance in guiding visually directed actions, leading many researchers to refer to it as the "how" pathway (Goodale & Milner, 1992).
• Think of it as the pathway responsible for determining "how" to interact with objects in the environment, such as reaching for a cup of coffee.

Ventral Pathway (The "What" Stream)

• The ventral pathway extends from V1 to the temporal lobe.
• It is famously known as the "What" stream, as it focuses on identifying and recognizing objects, their properties, and features (Ungerleider & Mishkin, 1982). The ventral stream integrates information about an object's shape, colour, texture, and other visual features to support object recognition and identification(Grill-Spector & Malach, 2004).
• This pathway is akin to telling you "what" the object is, like identifying that the cup of coffee contains your favorite latte.

Neural Basis of Dorsal and Ventral Pathways

Leslie Ungerleider and Mortimer Mishkin's research played a pivotal role in understanding the neural basis of these pathways (Ungerleider & Mishkin, 1982).

• Magnocellular Layers (LGN): These layers are associated with the dorsal pathway, processing motion and spatial information.
• Parvocellular Layers (LGN): These layers contribute to the ventral pathway, allowing for the identification of object features like shape, color, and texture.
•  Note that although magnocellular inputs contribute primarily to dorsal stream processing and parvocellular inputs contribute primarily to ventral stream processing, this distinction is not absolute. As visual information passes through the cortex, both streams receive and integrate information from multiple sources (Nassi & Callaway, 2009).

Double Dissociation from Brain Damage

Ungerleider and Mishkin's groundbreaking research included lesion studies that revealed a double dissociation between the effects of temporal and parietal lobe damage on visual processing (Ungerleider & Mishkin, 1982).

Temporal Lobe Damage

• Damage to the temporal lobe leads to difficulties in object recognition.
• Patients may exhibit visual form agnosia, where they struggle to identify objects, even though they can describe them in terms of shape, color, and other features. Importantly, these individuals are not blind. They can see the object's individual visual features but are unable to combine those features into a coherent visual form that would allow the object to be recognized (Milner et al., 1991).
• Temporal lobe damage also affects the ability to copy objects accurately.
• Individuals with visual form agnosia are not blind. Their eyes and early visual processing remain largely intact, allowing them to detect the presence of objects and perceive basic visual features such as color, brightness, orientation, and sometimes shape. However, they cannot integrate these features into a coherent representation that enables them to recognize or identify the object from vision alone.

Parietal Lobe Damage

• Damage to the parietal lobe results in difficulties with spatial processing and interactions with objects.
• Patients may experience optic ataxia, where they can identify objects but struggle to grasp and interact with them accurately. Their motor system is not weak or paralyzed; rather, the difficulty lies in transforming visual information into accurate reaching and grasping movements (Perenin & Vighetto, 1988).
• This damage primarily affects the "where" and "how" aspects of visual processing.

Double Dissociation

Together, visual form agnosia and optic ataxia provide a compelling illustration of a double dissociation. A double dissociation occurs when two related cognitive abilities can be independently impaired: one individual can perform cognitive task A but struggles with cognitive task B, while another individual shows the opposite pattern, performing task B successfully but struggling with task A. This complementary pattern of deficits provides strong evidence that the two cognitive functions rely on distinct neural systems rather than reflecting different levels of difficulty within a single process. Importantly, a double dissociation is considered much stronger evidence than a single dissociation because it demonstrates that each cognitive function can be selectively impaired while the other remains relatively intact (Shallice, 1988).

In visual form agnosia, damage to the ventral ("what") visual pathway impairs the ability to recognize or identify objects from vision alone, yet visually guided actions such as reaching toward an object, scaling the grip to its size, and orienting the hand appropriately for grasping remain largely intact. Conversely, in optic ataxia, damage to the dorsal ("how/where") visual pathway preserves the ability to recognize and describe objects but disrupts the accurate visual guidance of reaching and grasping movements (Perenin & Vighetto, 1988). Thus, the ability to recognize an object and the ability to interact with it are dissociable.

This double dissociation demonstrates that object recognition and visually guided action depend on partially distinct neural pathways that can be selectively impaired or preserved following brain damage. More broadly, double dissociations provide strong evidence in cognitive neuroscience that seemingly related cognitive functions are separable and supported by different neural substrates. This approach has become one of the most powerful methods in cognitive neuroscience for inferring the functional organization of the brain from patterns of preserved and impaired behaviour following brain injury (Shallice, 1988).

Binocular Rivalry and Continuous Flash Suppression (CFS)

Now, let's explore a fascinating technique called continuous flash suppression (CFS), which can render visual information unconscious by suppressing the ventral pathway while keeping the dorsal pathway active. More precisely, CFS greatly reduces conscious awareness of stimuli while still allowing some unconscious visual processing to occur, particularly within neural systems involved in visually guided action (Tsuchiya & Koch, 2005; Lin & He, 2009).

Determining Eye Dominance

Before we can understand CFS, it's crucial to determine a person's eye dominance. It turns out, just like hand dominance people have a dominant eye. One effective method for determining eye dominance involves the following steps:

1. Extend your hands in front of you (so your elbows are straight), creating a triangular gap between the pointer finger and thumb of each hand.
2. While keeping both eyes open, focus on an object through the gap.
3. Close your left eye. If the object disappears, you are left-eye dominant. If it remains visible, you are right-eye dominant.
4. Repeat the process, this time closing your right eye. If the object disappears, you are right-eye dominant.

Binocular Rivalry and Continuous Flash Suppression

CFS capitalizes on binocular rivalry, a phenomenon where each eye receives different visual information, leading to perceptual alternations between the two images (Blake & Logothetis, 2002). This can be achieved, for example, by presenting an individual with a superimposed image of a dog and an ice cream cone, each selectively visible

through differently colored filters (see Figure 5.1). Specifically, the image of the ice cream cone is perceptible when viewed through a blue lens but remains concealed when observed through a red lens. Conversely, the image of the dog becomes apparent when seen through the red lens but remains obscured under the blue lens.

Figure 5.1

In binocular rivalry each eye is presented a different image. Here the left eye is shown a picture of a dog and the right eye is shown a picture of an ice cream

"Binocular rivalry." by Kahan, T.A. is licensed under CC BY-NC-SA 4.0

When an individual wears specialized red/blue glasses, with the red lens covering one eye and the blue lens covering the other, each eye receives a distinct image input. What makes this phenomenon particularly captivating is that the individual's conscious experience fluctuates intermittently. At times, they become consciously aware of perceiving the dog, while at other times, they shift their conscious awareness to the image of the ice cream cone (Blake & Logothetis, 2002).

Continuous Flash Suppression: A Unique Version of Binocular Rivalry

1. A person wears red-blue 3D glasses with the red lens over their dominant eye and the blue lens over the non-dominant eye.
2. Randomly changing visual noise (static) is shown to the dominant eye.
3. In the non-dominant eye, a clear image, such as an object or scene, is presented.
4. Despite the clear image being shown to one eye, the viewer's conscious experience is dominated by the rapidly changing static in the dominant eye.
5. This effectively suppresses the person's awareness of the clear image presented to the non-dominant eye (Tsuchiya & Koch, 2005).

Priming Study: In 2008, Almeida, Mahon, Nakayama, and Caramazza conducted a priming study with CFS that revealed interesting findings. Participants were exposed to images under CFS, making it difficult to identify the images consciously. However, when asked to categorize a subsequent image as tools or animals, they exhibited priming effects only for tools (i.e., faster to respond to a tool that was preceded by a tool relative to a tool preceded by an animal even though the first object was not consciously seen). Since this finding did not occur for animals and since people have experience grasping tools but not animals (the how of visual processing), this suggests that the dorsal stream may be involved in this priming effect (Almeida et al., 2008).

Shape Processing in Continuous Flash Suppression: Recent research has challenged the notion that priming in continuous flash suppression reflects “how” level processing rather than information about the visual look of an object. Some studies indicate that in this state, our brains might primarily process the shape of objects rather than their meaning. For instance, priming effects were observed even when tools (long thin objects) were preceded by unrelated objects like snakes as long as the shape was the same. These results suggest that the processing might focus on the shape of the objects rather than how we will interact with the object (Sakuraba et al., 2012).

The extent to which semantic information can be processed under continuous flash suppression remains an active area of research. Some studies have reported evidence that the meaning of words can influence later processing even when the words are not consciously perceived, whereas other studies have found little or no evidence that semantic information survives suppression. These mixed findings suggest that unconscious semantic processing under CFS is more limited than originally believed and may depend on the specific task and experimental design (Prioli & Kahan, 2015; Heyman & Moors, 2014).

Modularity in the Brain

Modularity refers to the idea that certain behaviors or mental processes (e.g., face recognition) are served by specific brain regions (Fodor, 1983).

PET Imaging and Modularity:

Positron emission tomography (PET) further supported the idea of modularity in the brain. In a task that required participants to determine which of two images was a 90- degree clockwise rotation of an initial image, brain activity showed clear distinctions. Dorsal stream activity was prominent in the "where" pathway when determining spatial relationships (which of two random dot patterns was a 90-degree rotation of an initial random dot pattern). Similarly, ventral stream activity was prominent in the "what" pathway, when recognizing faces (which of two faces was a 90-degree rotation of an initial face) (Haxby et al., 1991).

Specialized Brain Areas for Faces and Places:

Visual perception is a complex process that relies on the coordinated activity of various specialized brain areas, each dedicated to processing specific types of visual information. Two critical brain regions that have garnered significant attention in the realm of visual processing are the Fusiform Face Area (FFA) and the Parahippocampal Place Area (PPA). The research conducted by Nancy Kanwisher and her colleagues has shed light on the specialized functions of these areas, sparking debates about the extent of their specificity.

Fusiform Face Area (FFA):

One of the pioneering figures in the field of face processing research is Nancy Kanwisher, whose work has illuminated the role of the Fusiform Face Area (FFA). The Fusiform Face Area (FFA) is located on the underside of the temporal lobe (in a region called the fusiform gyrus), with activity usually being stronger in the right hemisphere. This brain region is especially important for recognizing and distinguishing faces (Kanwisher et al., 1997). Studies utilizing functional magnetic resonance imaging (fMRI) consistently show heightened activity in the FFA when individuals view faces. This area not only responds to facial features but also plays a crucial role in facial recognition and discrimination. Kanwisher's research has provided compelling evidence for the specificity of the FFA (Kanwisher et al., 1997). When subjects are exposed to images of faces, there is a distinct and robust increase in FFA activity. This distinctiveness implies that the FFA is finely tuned to facial processing and serves as a dedicated neural module for this purpose. Moreover, damage to the FFA can result in a condition known as prosopagnosia, characterized by the inability to recognize faces (Barton, 2008), even those of close family members or friends.

Parahippocampal Place Area (PPA):

The Parahippocampal Place Area (PPA), is located in the lower, inner portion of the temporal lobe in both hemispheres. It responds strongly when we view places, buildings, landscapes, and other scenes (Epstein & Kanwisher, 1998). PPA activity significantly increases when individuals view images of landscapes, buildings, or other place-related stimuli. This area plays a pivotal role in spatial navigation and our ability to recognize and navigate through different environments (Epstein & Kanwisher, 1998).

Research investigating the PPA has demonstrated its specialization for place processing.

Studies utilizing fMRI consistently reveal heightened PPA activation in response to place-related visual stimuli. Moreover, individuals with damage to the PPA often experience difficulties in recognizing or navigating through spatial environments.

Debate Surrounding FFA and Its Generalization to Other Stimuli:

While the FFA's specialization for faces is well-established, there has been an ongoing debate about whether this area can process other types of visual information. Nancy Kanwisher's work primarily focused on faces, but the question arose: Can the FFA respond to complex visual stimuli other than faces?

Isabel Gauthier's research has provided significant insights into this debate. Her studies have examined whether individuals can become experts in recognizing non-face stimuli, such as greebles (novel objects), cars, or birds, and if so, whether the FFA plays a role in processing these stimuli. Gauthier's findings suggest that individuals who become experts in recognizing non-face stimuli exhibit FFA activation when exposed to these stimuli (Gauthier et al., 2000; Gauthier & Tarr, 2002), demonstrating that the FFA may not be as strictly specialized as previously thought.

Conclusion

In this chapter, we have ventured beyond the primary visual cortex (V1) to explore the intricacies of higher-level visual processing. We discussed the two major pathways out of V1 (dorsal and ventral) and the concept of modularity. As we conclude this chapter, we are left with a profound appreciation for the complexity and versatility of the human visual system, which continues to be a source of inspiration and exploration for researchers seeking to unlock the secrets of perception, cognition, and consciousness

References

Almeida, J., Mahon, B. Z., Nakayama, K., & Caramazza, A. (2008). Unconscious processing dissociates along categorical lines. Proceedings of the National Academy of Sciences, 105(39), 15214–15218. https://doi.org/10.1073/pnas.0805867105

Barton, J. J. S. (2008). Structure and function in acquired prosopagnosia: Lessons from a series of 10 patients with brain damage. Journal of Neuropsychology, 2(1), 197–225. https://doi.org/10.1348/174866407X214172

Blake, R., & Logothetis, N. K. (2002). Visual competition. Nature Reviews Neuroscience, 3(1), 13–21. https://doi.org/10.1038/nrn701

Epstein, R., & Kanwisher, N. (1998). A cortical representation of the local visual environment. Nature, 392, 598–601. https://doi.org/10.1038/33402

Fodor, J. A. (1983). The modularity of mind. MIT Press.

Gauthier, I., Skudlarski, P., Gore, J. C., & Anderson, A. W. (2000). Expertise for cars and birds recruits brain areas involved in face recognition. Nature Neuroscience, 3(2), 191–197. https://doi.org/10.1038/72140

Gauthier, I., & Tarr, M. J. (2002). Unraveling mechanisms for expert object recognition: Bridging brain activity and behavior. Journal of Experimental Psychology: Human Perception and Performance, 28(2), 431–446. https://doi.org/10.1037/0096-1523.28.2.431

Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action. Trends in Neurosciences, 15(1), 20–25. https://doi.org/10.1016/0166-2236(92)90344-8

Grill-Spector, K., & Malach, R. (2004). The human visual cortex. Annual Review of Neuroscience, 27, 649–677. https://doi.org/10.1146/annurev.neuro.27.070203.144220

Haxby, J. V., Grady, C. L., Horwitz, B., Ungerleider, L. G., Mishkin, M., Carson, R. E., Herscovitch, P., Schapiro, M. B., & Rapoport, S. I. (1991). Dissociation of object and spatial visual processing pathways in human extrastriate cortex. Proceedings of the National Academy of Sciences, 88(5), 1621–1625. https://doi.org/10.1073/pnas.88.5.1621

Heyman, T., & Moors, P. (2014). Frequent words do not break continuous flash suppression differently from infrequent or nonexistent words: Implications for semantic processing of words in the absence of awareness. PLoS ONE, 9(8), e104719. https://doi.org/10.1371/journal.pone.0104719

Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: A module in human extrastriate cortex specialized for face perception. Journal of Neuroscience, 17(11), 4302–4311. https://doi.org/10.1523/JNEUROSCI.17-11-04302.1997

Lin, Z., & He, S. (2009). Seeing the invisible: The scope and limits of unconscious processing in binocular rivalry. Progress in Neurobiology, 87(4), 195–211. https://doi.org/10.1016/j.pneurobio.2008.09.002

Milner, A. D., Perrett, D. I., Johnston, R. S., Benson, P. J., Jordan, T. R., Heeley, D. W., Bettucci, D., Mortara, F., Mutani, R., Terazzi, E., & Davidson, D. L. W. (1991). Perception and action in 'visual form agnosia'. Brain, 114(1), 405–428. https://doi.org/10.1093/brain/114.1.405

Nassi, J. J., & Callaway, E. M. (2009). Parallel processing strategies of the primate visual system. Nature Reviews Neuroscience, 10(5), 360–372. https://doi.org/10.1038/nrn2619

Perenin, M. T., & Vighetto, A. (1988). Optic ataxia: A specific disruption in visuomotor mechanisms. I. Different aspects of the deficit in reaching for objects. Brain, 111(3), 643–674. https://doi.org/10.1093/brain/111.3.643

Prioli, S. C., & Kahan, T. A. (2015). Identifying words that emerge into consciousness: Effects of word valence and unconscious previewing. Consciousness and Cognition, 35, 88–97. https://doi.org/10.1016/j.concog.2015.04.005

Sakuraba, S., Sakai, S., Yamanaka, M., Yokosawa, K., & Hirayama, K. (2012). Does the human dorsal stream really process a category for tools? Journal of Neuroscience, 32(11), 3949–3953. https://doi.org/10.1523/JNEUROSCI.3973-11.2012

Tsuchiya, N., & Koch, C. (2005). Continuous flash suppression reduces negative afterimages. Nature Neuroscience, 8(8), 1096–1101. https://doi.org/10.1038/nn1500

Shallice, T. (1988). From neuropsychology to mental structure. Cambridge University Press.

Ungerleider, L. G., & Mishkin, M. (1982). Two cortical visual systems. In D. J. Ingle, M. A. Goodale, & R. J. W. Mansfield (Eds.), Analysis of visual behavior (pp. 549–586). MIT Press.