Camera-based sensory substitution and augmented reality for the blind

« The vOICe Home Page

Listed below is the abstract of the invited opening keynote of the co-located  ACIVS 2011 conference (Advanced Concepts for Intelligent Vision Systems), August 22—25, 2011, and  ICDSC 2011 conference (5th edition of the ACM/IEEE International Conference on Distributed Smart Cameras), August 23—26, 2011, Ghent, Belgium. This presentation took place on Monday August 22, 2011, 09:00-10:00. A PDF file with the presentation slides is also available:  Camera-based sensory substitution and augmented reality for the blind

USB camera glasses for sensory substitution and sonic augmented reality Title

Camera-based sensory substitution and augmented reality for the blind

Speaker

Peter Meijer, Metamodal BV, Eindhoven, The Netherlands

Abstract

Rapid developments in mobile computing and sensing with smartphones open up new opportunities for augmenting our reality with information and experiences that our senses could not directly provide. One current trend is towards augmented reality applications based on location-based services (LBS) and computer vision. Apart from mass-market uses, there also arise new uses in niche markets such as technology for the blind.

Despite its more limited commercial value, I will in my talk discuss how this particular niche market is extremely interesting for bringing together research on man-machine interfaces, computer vision, brain plasticity, synesthesia, and even contemporary philosophy. It is also an area where fundamental research (e.g. on brain plasticity) may prove directly socially relevant through applications that are readily made globally available over the web, and that run on mass-market devices.

Hybrid applications convey via sound or touch the raw visual information from live camera views as well as semantic information for nearby items of interest, as recognized through computer vision or identified through location databases. Moreover, neuroscience research has in the past decade established that the visual cortex of blind people becomes responsive to sound and touch, thus adding some biological plausibility to the idea of creating non-invasive sensory by-passes in the form of sensory substitution.

(PDF file of slides) Camera-based sensory substitution and augmented reality for the blind

Copyright © 1996 - 2025 Peter B.L. Meijer