LO is a meta-modal operator for shape

Amir Amedi ¹, Felix Bermpohl ¹, Joan Camprodon ¹, Lotfi Merabet ¹, Peter Meijer ² and Alvaro Pascual-Leone ¹.

¹ Center for Non-Invasive Magnetic Brain Stimulation, Dept. of Neurology, BIDMC, Harvard Medical School, MA02215, USA.
² Philips Research, Eindhoven, The Netherlands. [Update: currently no longer working at Philips.]

We have previously hypothesized that part of the lateral occipital complex (LOC / LOtv) is processing and representing the geometrical shape of objects. This was based on the fact that LO is robustly activated by visual and tactile object recognition, sensory modalities in which shape information is rich (Amedi et al. 2002). Furthermore, LO is only negligibly activated by "auditory objects" (i.e. by the typical sounds made by objects) in which object recognition is generally based on associations rather than on shape information. If LOC / LOtv is indeed an operator for shape, we hypothesized that it would be recruited for processing auditory information if that is (somehow) rich in shape information. Visual-to-auditory sensory substitution devices (SSD) provide a unique opportunity to study such a situation systematically: visual images are captured by a camera and then transformed, according to an arbitrary algorithm, into soundscapes that contain shape information conveyed by sound. We report here fMRI results in two blind (one congenitally and one late blind) expert users of a visual-to-auditory SSD called "The vOICe". This system encodes a visual scene following 3 simple rules: 1) the vertical axis is represented by sound frequency; 2) the horizontal axis is represented by time and stereo panning; and 3) brightness is encoded by loudness (Meijer, 1992). In the current experiment we compared object recognition based on information arriving from different sources. The design included object recognition by touch (object palpation), auditory (e.g. typical sounds made by objects), visual-to-auditory transformed objects (using visual objects encoded into soundscapes), and corresponding low-level sensory controls.

Consistent with prior results, LOC / LOtv is robustly activated by object palpation, but shows significantly less, minimal activation to the typical sounds made by objects (though activation in these blind subjects is greater than that seen in sighted subjects). Critically, this area shows robust activation to the soundscapes of visual- to-auditory transformed objects (‘The vOICe’ condition), that is as prominent as that seen for the object palpation, and equivalent to what might be expected to visual object processing in sighted.