SmartSight for the Blind?

The vOICe for Windows translates images from your PC camera (webcam) into sounds that you hear via your stereo headphones, thereby targetting synthetic vision for the totally blind by means of a wearable computer. The vOICe for Windows has been available since January 1998. The vOICe approach was originally published in the IEEE Transactions on Biomedical Engineering, Vol. 39, No. 2, pp. 112-121, Feb 1992: P.B.L. Meijer, ``An Experimental System for Auditory Image Representations.'' This paper was next selected for reprint in the 1993 IMIA Yearbook of Medical Informatics, pp. 291-300. Awarded U.S. Patent 5097326, on an "image-audio transformation system", filed July 27, 1990: ``In a device for converting visual images into representative sound information especially for visually handicapped persons an image processing unit is provided with a pipelined architecture with a high level of parallelisum. An image is scanned in sequential vertical scanlines and the acoustical representatives of the scanlines are produced in real time. Each scanline acoustical representation is formed by sinusoidal contributions from each pixel in the scanline, the frequency of the contribution being determined by the position of the pixel in the scanline and the amplitude of the contribution being determined by the brightness of the pixel.''

• Literature on The vOICe approach:

  Meijer, P. B. L., ``An Experimental System for Auditory Image Representations,'' IEEE Transactions on Biomedical Engineering, Vol. 39, No. 2, pp. 112-121, Feb 1992. Reprinted in the 1993 IMIA Yearbook of Medical Informatics, pp. 291-300.

  Meijer, P. B. L., ``Cross-Modal Sensory Streams,'' invited presentation and demonstration at SIGGRAPH 98 in Orlando, Florida, USA, July 19-24, 1998. Conference Abstracts and Applications, ACM SIGGRAPH 98, 1998, p. 184.

  Meijer, P. B. L., ``Seeing with Sound for the Blind: Is it Vision?,'' invited presentation at the Tucson 2002 conference on Consciousness, Tucson, Arizona, USA, April 8-12, 2002. Conference research Abstracts, Journal of Consciousness Studies, 2002, abstract 187, p. 83.

  Fletcher, P. D., ``Seeing with Sound: A Journey into Sight,'' invited presentation at the Tucson 2002 conference on Consciousness, Tucson, Arizona, USA, April 8-12, 2002. Conference research Abstracts, Journal of Consciousness Studies, 2002, abstract 188, p. 83.

• For other useful literature, see another list of references. Visit the related projects page for other projects in which image-tosound mappings similar to or related to The vOICe mapping are employed. Also note that the EyeMusic approach (Yissum Research Development Company of the Hebrew University of Jerusalem) is similar to the SmartSight approach of using music to convey visual images.

• Publications relating to SmartSight:

  Proc. Royal Society of London Series B - Biological Sciences, Vol. 266, pp. 2427-2433, 1999.

  Cronly-Dillon, J. R., Persaud, K. C., Blore, R. W., Gregory, R. P. and Harvey, K., ``Blind subjects analyse photo images of urban scenes encoded in musical form,'' Investigative Ophthalmology and Visual Science, 41, 3862B960, 2000.

  Cronly-Dillon, J., Persaud, K. and Gregory, R. P. F., ``Vision parallax (translated into musical form) enables blind subjects to estimate relative distance between visual targets,'' Journal of Physiology - London, Vol. 526, 152P, 2000.

  Cronly-Dillon, J. and Persaud, K., ``Blind subjects analyse visual images encoded in sound,'' Journal of Physiology - London, Vol. 523, 68P-69P, 2000.

  Cronly-Dillon, J., Persaud, K. C. and Blore, R. ``Blind subjects construct conscious mental images of visual scenes encoded in musical form,'' Proc. Royal Society of London Series B - Biological Sciences, Vol. 267, pp. 2231-2238, 2000. Available  online (PDF file).