Chapter 2: Sensory Memory

Introduction

Sensory memory is a crucial yet often overlooked component of the human memory system. Unlike short-term or long-term memory, sensory memory involves the initial brief storage of sensory information. This chapter will explore the nature of sensory memory, its different types, and significant research that has enhanced our understanding of this memory system.

What is Sensory Memory?

Sensory memory is the initial storage system for sensory stimuli from our environment. Each of our five senses—sight, hearing, touch, taste, and smell—has a corresponding sensory memory system. These systems capture a large amount of sensory information for a very brief period, allowing for the initial processing and potential transfer to short-term memory.

The two most studied types of sensory memory are:

• Iconic Memory: Visual sensory memory
• Echoic Memory: Auditory sensory memory

There is also evidence suggesting the existence of sensory memories for tactile, olfactory, and gustatory information, although these are less studied.

Iconic Memory: Visual Sensory Memory

Iconic memory is the term used for visual sensory memory. This type of memory was extensively studied by George Sperling in the 1960s. Sperling's experiments revealed that iconic memory can hold a vast amount of visual information for a very short duration.

Sperling's Experiment

To demonstrate iconic memory, imagine being shown a grid of 12 letters (three rows of four columns) for a brief moment. When the display disappears, you try to recall and write down as many letters as you can remember. Typically, people can recall about four to five letters, or roughly one-third of the total display.

Sperling introduced two key concepts through this type of experiment:

• Whole Report Condition: Participants try to recall the entire array of letters.
• Partial Report Condition: Participants are cued to recall only a specific row of letters, indicated by a high, medium, or low-pitched tone presented immediately after the display disappears.

In the whole report condition, participants could recall about 33% of the letters. However, in the partial report condition, participants could recall about 76% of the letters from any cued row, suggesting that the entire display was initially available in sensory memory. This finding indicated that iconic memory could hold much more information than previously thought (certainly more than 33%) but that this information decayed quickly.

Decay vs. Output Interference

Sperling's results sparked a debate on whether the rapid loss of information in iconic memory was due to decay over time or output interference where the act of recalling one item causes other items to be lost from memory.

To test these hypotheses, Sperling manipulated the delay between the display disappearing and the tone cue. If decay caused the loss, performance would decline with longer delays. If output interference was the cause, performance would remain constant regardless of delay. Sperling found that performance did decline with longer delays, supporting the decay theory. In addition, since performance reached 33% after 250 ms or so, the data indicate that iconic memory only lasts for a quarter of a second. This is shown in Figure 2.1.

Figure 2.1. This figure shows the number of letters available (y axis) in part report (PR) and whole report (WR) as a function of cue delay. Critically, accuracy decreases as the delay increases and the part-report superiority effect is eliminated after 250 ms

"Sperling’s basic pattern of data." by Kahan, T.A. is licensed under CC BY-NC-SA 4.0

Later research on iconic memory found that people are not better at recalling a portion of the display (part report) relative to the entire array (whole report) when they are cued to recall letters or numbers. i.e., There is no part-report superiority effect for letters vs. numbers. Since the distinction between letters and numbers is based on prior learning that is stored in long-term memory and is not a physical feature of the visual stimulus, this result supports the claim that iconic memory contains raw, pre-categorical, information.

Characteristics of Iconic Memory

Based on Sperling's and subsequent research, the key characteristics of iconic memory include:

• Capacity: Iconic memory can hold a large amount of visual information, potentially all the information in the visual scene.
• Duration: The information in iconic memory decays within approximately 250 milliseconds (a quarter of a second).
• Nature: Iconic memory is pre-categorical, meaning it holds raw sensory data that have not yet been processed into meaningful categories like letters or numbers.

Additional Research on Iconic Memory

Further studies, such as those by Loftus, Johnson, and Shimura, provided more insights into iconic memory. They demonstrated that iconic memory allows us to continue extracting visual details for a brief period (100 ms, or so) even after the visual stimulus is removed. Their experiments showed that participants could recall more details about a briefly presented image if there was a brief delay before a masking pattern was presented, highlighting the transient nature of iconic memory. Specifically, people were able to recall the same number of details in a delayed offset condition as they would if the image had been shown for 100 ms longer but was immediately masked. This pattern indicates that people can continue to extract information from an iconic afterimage that fades rapidly.

Criticisms of Iconic Memory Research and a Possible Response

Haber criticized iconic memory research for lacking ecological validity, suggesting that the experimental conditions under which iconic memory is studied are too artificial to be relevant in everyday life. He likened the findings to discovering that the visual icon could only be useful for reading during a lightning storm, implying that such conditions are so rare and unnatural that the results have little practical application. His critique emphasized that the fleeting nature of the visual icon, as typically studied, might not be relevant to everyday visual processing where visual input is more continuous and less abruptly disrupted.

However, research has shown that people do not take in visual input during saccades, which are rapid eye movements that happen several times a second as we scan our environment. This phenomenon is related to change blindness, where significant changes in a visual scene often go unnoticed if they occur during a saccade (see Figure 2.2). The concept of iconic memory can have ecological validity in this context, as it may help maintain a stable and continuous perception of the world by briefly holding onto visual information across saccades. This ability allows our visual system to integrate information smoothly and efficiently, supporting our experience of a stable environment even though our eyes are constantly moving.

Figure 2.2. This figure illustrates change blindness. In a spot the difference task you must switch your attention from one image to the other to find what has changed and this is often quite difficult. In the image to the left more of the trees and flowers on the right side of the image are visible, there are trees on the left in the distance that are not present in one image but are in the other, and there is a bird on the ground in the right image that is not present in the left image. This image was generated by ChatGPT from the prompt “Can you generate two nearly identical images one to the left of the other of an outdoor scene where there are a small number of differences between the two images. For example, maybe an object could be larger in one scene or missing from a scene. This will be used in a spot the difference task where a viewer must identify what is different between the two images.”

"Change blindness example." by Kahan, T.A. is licensed under CC BY-NC-SA 4.0

Echoic Memory: Auditory Sensory Memory

Echoic memory is the auditory equivalent of iconic memory, responsible for holding auditory information for a brief period. It allows us to retain sounds long enough to process and understand them. Echoic memory typically lasts longer than iconic memory, with a duration of up to a few seconds.

Characteristics of Echoic Memory:

• Duration: Echoic memory lasts longer than iconic memory, approximately up to 4 seconds. This estimate was determined using an auditory analog of Sperling’s part-report superiority task. In this task, known as the three-eared man participants heard letters presented to the left ear, right ear, or both ears (center ear). A cue indicated which ear to recall (part report) or to recall the entire array of letters (whole report). As was found for visual information there was a part-report superiority effect that quickly decayed; however unlike iconic memory the part-report superiority effect lasted for roughly 4 seconds.
• Capacity: It can store a significant amount of auditory information briefly.
• Evidence:

• Part-Report Superiority Effect: Similar to iconic memory's partial report superiority effect, where people can recall more details when cued selectively, echoic memory also exhibits better recall performance with partial cues compared to whole report.
• Modality Effect: Echoic memory shows an advantage for auditory stimuli over visual stimuli, especially towards the end of rapidly presented lists of items. In other words, the recency effect (which is elevated memory for information presented at the end of a list) is greater when the information is presented in the auditory modality relative to the visual modality.
• Suffix Effect: Introducing an auditory stimulus (suffix) at the end of a list disrupts recall of earlier items, indicating the presence of active auditory memory storage that can be overwritten by new auditory input. In other words, the recency effect is greatly reduced when an auditory stimulus is presented at the end of a list.

Pre-Categorical Acoustic Storage: Proposed by Crowder and colleagues to explain how auditory information in echoic memory is stored before categorization into meaningful units occurs in short-term memory. According to this theory echoic memory, which they referred to as pre-categorical acoustic storage will supplement other memory systems. When a person is presented with a rapid list of words the first items are moved into short-term memory and this is responsible for the primacy effect. However, if the words are presented in the auditory modality then the last few words are still available in echoic memory (pre-categorical acoustic storage) and this is responsible for the recency effect.

Critiques and Validation:

• Pre-Categorical Nature: Echoic memory primarily stores acoustic properties rather than semantic or meaningful content, validated through experiments showing similar suffix effects regardless of semantic similarity. The suffix effect is not larger when the suffix is from the same category as the list items relative to when the suffix and list items are from different categories. Instead, the only thing that seems to affect the magnitude of the suffix effect is the physical similarity between the list items and the suffix (e.g., same-voice suffixes will produce a larger suffix effect than different-voice suffixes). As another example, participants were presented with lists of homophones (e.g., "buy," "by," "bye") and non-homophones ("dog," "up," "it"). When read aloud, participants exhibited a recency effect with non-homophones but not homophones, suggesting that echoic memory retains phonetic distinctions rather than the meaning of the words.
• Acoustic vs. Semantic Processing: Echoic memory is more sensitive to acoustic properties, such as pitch and tone, rather than semantic meaning.

Real-world examples:

• The temporal dynamics of speech perception may be an important factor that affects echoic memory. Specifically, since auditory information unfolds over time it may be evolutionarily advantageous to be able to hold auditory information for a longer period and this may help with language processing.
• You may have experienced a situation where someone is talking to you, and you initially respond with "What?" because you think you didn't hear them. However, before they repeat themselves, you say "Never mind" because you realize you continued to hear them in your mind. This occurs because echoic memory lasts for roughly 4 seconds, allowing you to continue processing auditory information for a short period.

Problems for the theory of pre-categorical acoustic storage

Ayres et al. (1979) conducted an experiment that challenges the conclusion that echoic memory is pre-categorical. In this study, all subjects heard a trumpet note before recall of a list of words, but there were two different conditions based on the instructions given:

1. Person Instructions: Subjects were told, "When you hear the person say ‘wa’, recall the words."
2. Trumpet Instructions: Subjects were told, "When you hear the trumpet tone ‘wa’, recall the words."

Critically, all participants heard the exact same list of words and the exact same ‘wa’ sound. The study found that the suffix effect occurred for the "person" instructions but not for the "trumpet" instructions. This suggests that the way people interpret the suffix (whether it is perceived as speech or a non-speech sound) influences the extent of the interference with recall, with speech sounds producing a stronger suffix effect. This is problematic for the theory of pre-categorical storage because the acoustic information (i.e., the physical suffix) was the same; instead, the difference was based on how people interpreted the meaning of the sound.

Contrary to initial assumptions, auditory stimuli do not exclusively dominate echoic memory. Studies involving American Sign Language (ASL) lists that are rapidly presented to deaf participants showed robust primacy and recency effects, challenging the notion of echoic memory as purely acoustic. This suggests a broader sensory storage mechanism encompassing non-auditory inputs.

Further research found that silently presented lip-read lists will also produce robust primacy and recency effects. This is problematic for the theory of pre-categorical acoustic storage because the information was not acoustic (i.e., no sound) yet the same recency effect emerged. Additionally, a silently lip-read suffix presented at the end of such a list will produce a robust suffix effect which is problematic for the idea that echoic memory is acoustic.

Interactions Between Visual Information and Auditory Perception: The McGurk Effect

The McGurk effect further illustrates the interplay between visual and auditory processes. When the auditory input "my bab pop you to brive" is paired with the visual input of someone mouthing "my gag kok you ku grive," the brain fuses these conflicting signals into a coherent perception of "my dad taught you to drive." This occurs because the brain integrates both auditory cues (what is heard) and visual cues (what is seen on the speaker's lips) to create the most plausible interpretation of speech. The McGurk effect highlights that visual information, such as lip movements, influences how sounds are perceived. Moreover, when people see someone silently mouthing words, they may automatically activate the corresponding phonemes in their mind, even without sound, showing that speech perception is a multisensory process involving both hearing and seeing.

Conclusion

Sensory memory plays a vital role in our ability to perceive and interact with the world. While iconic and echoic memory are the most studied types, sensory memory systems for touch, taste, and smell also exist and hold potential for further research. Understanding sensory memory provides a foundation for exploring more complex memory systems, such as short-term and long-term memory, which will be discussed in subsequent chapters.