Authors

Jung-Kyong Kim ¹ and Robert J. Zatorre ¹.

¹ Cognitive Neuroscience Unit, Montreal Neurological Institute, McGill University, Canada.

Abstract

Visual substitution learning of abstract shapes via auditory input

The notion behind visual substitution is to replace vision by another sensory input while conveying the essence of the visual information. The current study investigated tactile learning using auditory input in sighted subjects. We used the visual substitution algorithm called the ‘vOICe’ which translates visual images into soundscapes (Meijer, 1992), but applied it to tactile stimuli. Over the course of approximately three weeks, subjects participated in nine ~2-hour training sessions during which they learned the relationship of various abstract shapes to their corresponding converted sounds. In order to prevent the use of vision, subjects were blindfolded throughout each session. They used their right hands to explore a tactile shape that was embossed on a piece of paper while simultaneously hearing the sound corresponding to the given shape. Subjects were evaluated with forced-choice recognition tasks at the end of every three training sessions using both the items encountered during training and new items designed to test for generalization. Significant improvement was observed over time in recognizing the corresponding converted sounds for both old and novel shapes, suggesting generalization of learning of shapes via auditory input. In addition, at the end of the three-week training period, subjects were given a visual task where they were to recognize visual images, seen for the first time, corresponding to converted sounds. High accuracy on this task was observed even though the feedback subjects received throughout training was solely tactile in nature, suggesting crossmodal transferability of the substitution learning from the tactile to the visual modality.