« The vOICe Home Page
« The vOICe for Windows
Archived English translation by Karen Chung of the original April 25, 2003 German article "Sehen mit dem Ohr" by Janko Röttgers (copyright © Frankfurter Rundschau).
On the trail of the film hero "Daredevil": Software for the blind transforms visual images into acoustic signals.
By Janko Röttgers.
Matt Murdock has been blind since childhood. He embodies the prototype of the honest lawyer, turning down lucrative cases, and preferring to fight pro bono for the rights of the poor and weak. At night, however, Murdock becomes Daredevil, the fearless superhero. His blindness has compensated him with an extremely well-developed sense of hearing. Daredevil sees - in the full sense of the word - what he hears. This is a brief outline of the story of the superhero epic Daredevil, which is currently playing in theaters. The blind lawyer is personified by Ben Affleck.
The acoustic sensory impressions sound like a futuristic mixture of computer animations and night-vision images. If Dutch engineer Peter Meijer is to be believed, these sensory impressions have nothing to do with superhero magic. Meijer has developed a computer program that should enable every blind person to have acoustic vision.
Meijer offers the program, which he calls "The vOICe", free of charge over the Internet. Those interested in exploring their surroundings will need a portable computer and a simple Webcam. Some users have also put together a professional outfit: their notebook computer is stashed in a backpack, the camera is hidden under glasses, and a headset with micorphone can be used to give oral computer commands. Thus outfitted, they can use the system in public without looking like a cyborg. Others simply use the program on a home computer to view such things as stock market charts auditorily.
No matter whether it's used with a Webcam or to view stock market charts, The vOICe works on the same principle, that is, it translates visual impressions into acoustic ones. The image is scanned from left to right and converted into a multi-toned sythesizer kind of sound. An object can be located based on the variables of time and pitch, and the volume of the sound represents the brightness of an object. A simple horizontal white line on a black background becomes a tone that maintains the same pitch; in contrast, complex images sound like experimental electronic music from the 70s.
Someone hearing these sounds for the first time will have a hard time
imagining that The vOICe can in fact facilitate better spatial orientation
for the blind. "A few simple ways of using the program can be learned in
minutes or hours," Meijer says. Someone seeking to rely on The vOICe as a
complete substitute for vision will have to train for a long time. "I
often compare the effort required in the learning process with learning a
foreign language," explains Meijer, though he hastens to add that "probably even
this analogy falls apart in places."
We still have way too little understanding of how the brain processes
visual information. For this reason, Meijer conceives of his software as a
kind of neuroscientific field experiment. An experiment that is much less
risky for his participants than other methods. "Instead of implanting
electrodes in the retina, in optical nerves, or even in the center for
sight in the brain, The vOICe requires no surgery," Meijer says. In addition,
theoretically it even offers a higher image quality than the still very
primitive implants.
Users of the program exchange experiences over a mailing list associated with Meijer's Web site. One user writes there of how he saw pictures of his family and dogs for the first time: "This changed world is very new for me, but it is wonderful, exciting, and makes me happy. I feel a childlike sense of wonder." Others report that after a period of learning, they perceive black-and-white images. There is also talk of perception with a greenish cast. Others, after a few months, described first impressions of experiencing spatial depth.
So can everyone in fact develop a kind of Daredevil sense of sight with
"The vOICe"? What sounds like science fiction is thoroughly in the realm of
the possible for Californian philosopher and neuroscientist Alva Noë. He
speaks with conviction when he says that "we can see even without eyes and a
visual center of the brain." Scientists still disagree over why the
replacement of one sensory pathway by another can work. But Noë is confident
that "it is to a considerable extent possible."
What Noë finds so intriguing about The vOICe is the questions it raises
about the functioning of natural sight. "Interestingly, the way the parts
of our brain that are responsible for vision, touch, hearing and so forth
work is practically identical. Why then is one part responsible for images,
another for sounds or tactile perceptions? According to Noë's theory, it
is not just the optical nerves alone that are responsible for processing
information into images. What is important is the context in which the
perception emerges, as are the movements connected with it.
The Heinrich-Heine University in Düsseldorf is also on the trail of these interrelationships. In an experiment commissioned by their Institute for Experimental Psychology, people without visual handicaps tested the system for three weeks. This was done to test whether the system brought about any measurable changes in the brain.
Amid all the optimism, neither Peter Meijer nor Alva Noë wishes to awaken any false hopes. "Our bodies are unusually precise instruments. Evolution has bestowed us with extraordinarily refined means to perceive the world," says Noë. "Artificial methods of sensory perception and substitution are by comparison extremely simplistic." In other words, for supersensory powers we must continue to rely on the dream factories of Hollywood.
Source URL of original German publication by Frankfurter Rundschau online: