Amir Amedi ¹, William Stern ², Ella Striem ¹, Uri Hertz ¹, Peter Meijer ³, and Alvaro Pascual-Leone ².

¹ Department of Physiology, Faculty of Medicine, The Hebrew University of Jerusalem, Israel.
² Berenson-Allen Center for Noninvasive Brain Stimulation, Department of Neurology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, USA.
³ NXP Semiconductors, Eindhoven, The Netherlands.

A what/where visual-to-auditory sensory substitution fMRI study:
Can blind and sighted hear shapes and locations in the visual cortex?

 Abstract

In sensory substitution devices (SSDs), visual information captured by an artificial receptor is delivered to the brain using non-visual sensory information. Using an auditory-to-visual SSD called 'The vOICe' we find that blind achieve successful performance on object recognition tasks, and specific recruitment of ventral and dorsal 'visual' structures for shape versus localization tasks. Comparable recruitment was observed also in sighted learning to use this device but not in sighted learning arbitrary associations between sounds and object identity. These results support the notion that visual structures can support relevant computations in the auditory modality if subjects can extract relevant information and proper training is applied. We also find using phase locking Fourier Techniques an array of topographic maps which can serve as a basis for such audio-visual integration. These results suggest 'The vOICe' can be useful for blind individuals' daily activities but it also has a potential use to 'guide' visual cortex to interpret visual information arriving from prosthesis.

[Supported by grant R21-EY0116168 (to APL), and HFSP LFT and CDA awards (to AA).]