Amir Amedi ¹, Felix Bermpohl ¹, Joan Camprodon ¹, Sharon Fox ¹, Lotfi Merabet ¹, Peter Meijer ² and Alvaro Pascual-Leone ¹.

¹ Center for Non-Invasive Magnetic Brain Stimulation, Dept. of Neurology, BIDMC, Harvard Medical School, MA02215, USA.
² Philips Research, Eindhoven, The Netherlands. [Update: currently no longer working at Philips.]

Objective

Restoration of sight in the blind imposes clinical and scientific challenges. Despite intensive efforts, restoration of truly functional vision using neuroprostheses has not been achieved. We have suggested that a major reason for this is that the blind’s brain undergoes profound plastic changes and we do not know enough about how to communicate with this changed cortical organization in order to generate meaningful visual perception (Merabet et al. 2005). We propose that sensory substitution devices (SSD) can play a major role towards such a goal on two fronts:

1. Be of use in daily life for blind people to perceive and recognize visual input transformed for processing by their intact senses;
2. 'Guide' the visual cortex to 'read' (i.e. recognize) and interpret the visual information arriving from a visual prosthesis.

Figure 1: Activation pattern in a congenitally blind subject, proficient user of The vOICe in the shape versus location experiment. Two geometrical shapes were presented in two locations: upper or lower visual field. In the shape condition the subject had to judge whether the two shapes were similar or different regardless of their location. In the location condition he had to report whether the two stimuli were in the same position regardless of shape. Preference for shape versus location was found in associative auditory areas as well as a constellation of visual areas. The visual activation was focused on right ventral areas (including ventral V1/V2).

Methods

We report here findings of behavioral and fMRI results in congenitally and late blind expert users of a visual-to-auditory SSD called 'The vOICe'. This system encodes the different aspects of a visual scene (brightness and spatial location) using auditory information ('sound-scapes') based upon 3 simple rules: 1) the vertical axis is represented by frequency; 2) the horizontal axis is represented by time and stereo panning; and 3) brightness is encoded by loudness (Meijer, 1992). In experiment 1, we studied shape versus location processing of visual geometrical shapes transformed into The vOICe format (Figure 1). In experiment 2, we compared object recognition using The vOICe with auditory and tactile object recognition.

Results & Discussion

The results suggest that a right hemisphere lateralized network of areas participates in processing auditory soundscapes. Interestingly, differential activation of ventral V1/V2 is associated with shape versus location decisions using The vOICe (Figure 1). Conclusions: The results suggest that the ventral visual stream is recruited to process shape information, while posterior parietal cortex contributes to location information in proficient blind users of The vOICe. We also present differences obtained between tactile and soundscape-based object recognition as well as inter-subject differences. The data is discussed within the context of visual restoration and rehabilitation efforts.

References & Acknowledgements

We thank our blind subjects for their insights and enthusiastic participation.