The vOICe for Windows

« The vOICe Home Page
« The vOICe Web App
« The vOICe for Android

Learn to see

Parked car scene (88K WAV or 18K MP3 soundscape)

The vOICe for Windows (The vOICe Learning Edition) for Windows-95, 98, ME, NT, 2000, XP, Vista, Windows 7, 8, 10 and Windows 11

What does it do? The vOICe for Windows translates arbitrary video images from a regular PC camera into sounds. This means that you can "see" with your ears, whenever you want to. Now step beyond your computer screen and screen reader and try this camera-based "scene reader". With a netbook, tablet or notebook PC you can even go mobile. How well you can learn to interpret the image sounds is something that only you can find out, but now you can indeed find out and learn through this software, for free! At the very least you may feel more connected to your visual environment. To what extent seeing with sound will not only find practical uses but through extensive usage also lead to visual or vision-like experiences is not known. Will you be among the pioneers? It is not for the faint of heart: just like in learning a foreign language, or mastering a musical instrument, or learning to read Braille by touch, at least months of training effort should be expected before it begins to feel second nature.

Is it an augmented reality game or a serious tool? It can be both, depending on what you want it to be. Some blind people wear it daily with a wearable setup to see/hear their environment as they go around, while other blind people (blind from birth) use it to experience for the very first time what vision is like, and thus learn more about vision through uncensored first-hand experience. Some late-blind people use it to refresh their memories of what things looked like. Still others use it to read charts, drawings or graphs. You decide. Blind people using or interested in using The vOICe can also join The vOICe user group (mailing list), listen to a twelve-minute MP3 streaming audio demonstration of The vOICe by blind presenter Jonathan Mosen on  ACB Radio in June 2000, or listen to the personal experiences of Pat Fletcher, one of the first blind users of The vOICe, in a twenty-three minute presentation at the  Tucson 2002 conference and in a twenty-two minute feature in the  CBC Radio One science program Quirks & Quarks of April 2005. Congenitally blind user Pranav Lal talks about his use of The vOICe in an  Infotech Audio Magazine interview in June 2007, while early blind user Brian Shaw talks about his use of The vOICe in a  WYPL radio interview in January 2008. In June 2012, Brian Shaw and Pranav Lal gave an extensive  interactive presentation on The vOICe at the Cisco Academy for the Vision Impaired (CAVI).

Video clips: TechTV (US television, 2002), WDR/3Sat (German television, 2003), Teleac/NOT (Dutch television, 2004), Channel 10 (Israeli television, 2009).

How does it work? There are three simple rules in the general image to sound mapping of greyscale camera images, each rule dealing with one fundamental aspect of vision: rule 1 concerns left and right, rule 2 concerns up and down, and rule 3 concerns dark and light. The actual rules of the game are

1. Left and Right.

   Video is sounded in a left to right scanning order, by default at a rate of one image snapshot per second. You will hear the stereo sound pan from left to right correspondingly. Hearing some sound on your left or right thus means having a corresponding visual pattern on your left or right, respectively.

2. Up and Down.

   During every scan, pitch means elevation: the higher the pitch, the higher the position of the visual pattern. Consequently, if the pitch goes up or down, you have a rising or falling visual pattern, respectively.

3. Dark and Light.

   Loudness means brightness: the louder the brighter. Consequently, silence means black, and a loud sound means white, and anything in between is a shade of grey.

In other words, The vOICe scans each camera snapshot from left to right, while associating height with pitch and brightness with loudness. All of this means, for example, that a straight bright line on a dark background, running from the bottom left to the top right, sounds as a tone steadily increasing in pitch: ooiieep. Two bright lines give two tones. Three distinct bright dots sound as three short beeps, and so on. Although the rules are simple, real-life images often give very complex sounds, because there is so much to be seen. The vOICe for Windows generates "soundscapes" with a 176 × 64 resolution by default, thereby making it roughly comparable to a retinal implant or brain implant with 10,000 electrodes. plain text mini-tutorial

Example sounds: Because of the generality of the approach, applications can be as varied as orientation with respect to a wall with a gate or reading a purely graphical oscilloscope display. On this site you can also find sound samples in MP3 format, further illustrating visual orientation, seeing/hearing a printed graph, a parked car, the US flag, the planet Saturn or an Access Symbol, and even watching television. The vOICe indeed offers you universal accessibility!

The vOICe for Windows can be used for many educational purposes, for instance by providing full access to graphs and figures in books, to graphics displayed on a computer screen or to traces of an oscilloscope. You simply point the camera to anything of interest and move the camera around to listen to whatever comes in view. In using a head-mounted camera, it can be used to learn good posture and gait, avoiding head drooping. The vOICe for Windows offers a unique and rich learning experience. In mobile usage, it is advised to practise extensively in a safe, familiar home environment. Obviously it may seem that it is then only telling you what you already know, but interpretation for extended situational awareness first needs to become second nature and become integrated with your other senses - including your natural hearing and touch.

Try it for yourself!

The next generation of The vOICe auditory display software is now available for free evaluation, offering

A walk in the garden wearing The vOICe (2010)

Picking up objects with The vOICe (MetaModal LLC, 2010)

Synthetic vision based on Truly visual input Hifi stereo sound from the auditory display Simply download and run: Easy installation Built-in speech output for the main functions A mute button: You decide what you want, and when Many advanced options, including stereo vision, digital zoom, slow-motion, fast-motion, negative video, a built-in scientific graphing calculator and integrated e-mail to send camera snapshots Integrated browser to see/hear images on the web Add-ons: MP3 audio encoding, JPEG and TIFF image support Open interface for adding third-party recognition engines for OCR, face and object recognition, as well as eye-tracker interfacing support and access to unmodified mainstream games.

The vOICe for Windows is not about some sonar device: you can see/hear any visual item, including photographs, drawings, signs, or pictograms. In fact, every visual shape gives a correspondingly unique sound, and The vOICe for Windows lets you see/hear visual perspective, doorways, buildings, trees, furniture, or... you name it! It may also serve as a warning system to help people with severe tunnel vision (e.g., due to Retinitis Pigmentosa, or caused by laser treatments of diabetic retinopathy) or hemianopia (visual field loss, e.g., due to stroke), by getting auditory feedback on changes in the visual periphery insofar as these occur within the camera view. Thus one may detect items of potential interest (e.g., obstacles or door posts) that might otherwise be missed for lack of peripheral vision. Upon hearing sound pattern changes, one may redirect the gaze and use normal central vision to determine what is causing those changes, and then react or ignore. Color identification is included as well, using human speech to turn The vOICe system into a kind of "talking camera".

How to install: Basically, you just

1. Connect any webcam to your PC after installing its drivers.
2. Download The vOICe for Windows software, and run it.

The vOICe software will automatically detect your camera and you can then immediately see/hear the live view from your camera via your stereo headphones. That's all there is to it! The vOICe supports a highly modular approach in both hardware and software: you choose the hardware/software combination that serves your interests best while matching your budget.

The table below gives much more detail about various issues and options concerning software and hardware requirements and alternatives.

Software and Hardware Requirements Blind people interested in using The vOICe can join the seeingwithsound user group to ask for the latest information on suitable hardware and software. Users of The vOICe assemble their own setups. Complete pre-assembled setups are not (yet) for sale

The vOICe for Windows software. Free and fully functional evaluation copy now available for download from this site as voice.exe    (version 2.12) If you experience download problems, please report. [An old 32-bit version voice1.94.exe version 1.94 from 2012 is still available for older systems.] Please read the disclaimer, and note that good quality stereo headphones make The vOICe soundscapes far less obtrusive. Optional free add-on modules exist for MP3 audio encoding, and JPEG, TIFF image support, as well as for OCR and eye-tracker support (source code). The download is less than 1 MB in size, and contains no spyware, adware or other malware. The vOICe for Windows software can if desired be run directly from a USB stick, such that you can run it on any PC without installation. See the update history if you are considering downloading an upgrade from an earlier version. If you experience problems in running this software, please read the FAQ. Specifically, newer cameras increasingly often require MJPEG support that must be installed separately. You can read a few accounts written by blind users of this software, and read some self-training suggestions. The vOICe can be used in combination with a screen reader like the free  NVDA or the commercial  JAWS screen reader.

Microsoft Windows: The vOICe for Windows software runs under Microsoft Windows-95, Windows-98, Windows Millennium, Windows NT 4.0, Windows-2000, Windows XP, Windows Vista, Windows 7, 8, 10 and 11. Note: although The vOICe software runs on many versions of Microsoft Windows, some brands or types of PC cameras may be designed to run only with a subset of these operating system versions. Therefore, before buying a PC camera, please make sure that it indeed has an appropriate driver for your operating system by checking the vendor's camera specifications.

PC camera compatible with Microsoft "Video for Windows" (VfW). Almost all regular USB  webcams are fine, including those that support the newer Microsoft WDM standard, because WDM contains a wrapper for the Video for Windows standard. Note that only one video application can use a camera at any given time, so you may need to quit any other video application software that comes with your camera, or else The vOICe will report that it cannot connect to your camera. A currently recommended and affordable head-mounted camera solution is the use of USB camera glasses with webcam function. These are video sunglasses that offer a consistent alignment of the camera with the head by having the tiny camera hidden in the nose bridge of sunglasses. The camera can otherwise function as a regular USB webcam, and the sunglasses form factor ensures an unobtrusive appearance. In all cases, use of a head-mounted camera is strongly recommended for most consistent sensory feedback, while use of a wide-angle lens makes it easier to maintain a mental map of the environment. Whether you are blind or not, beware of possible legal and security implications of buying and wearing a (concealed) camera in your country. The relevant laws differ per country.

Computer: a 300 MHz or faster netbook, tablet or notebook PC is recommended, with 1 GHz or faster for best results. Regular desk-top PC's are also fine for a first evaluation, but then you obviously cannot walk around with it to immersively experience the intended full interactivity of seeing with sound. A backpack with a netbook or tablet will easily weigh much less than the 10-15 percent of body weight that is often quoted as a safety guideline for school children's backpack load ( APTA). Make sure that your PC includes A regular soundcard, and stereo headphones. Do not use loudspeakers, nor 5.1 surround headphones, because The vOICe sounds are designed and rendered for normal headphones. To minimize obstruction of environmental sounds - also for compatibility with echolocation - you can consider wearing  Shokz bone conduction headphones, or  AirDrives. However, you can also wear regular open headphones just above or just behind your ears to much the same effect. Suitable connectors for a PC camera. Most popular are currently USB cameras. The netbook, tablet or notebook PC can be used running in a backpack if sufficient ventilation is ensured.

Just to give you a concrete idea as a starting point, without necessarily implying that this is the best choice: an actual mobile setup may consist of any netbook or tablet running inside a well-vented backpack, supplemented by USB camera glasses, wide-angle lens, and a stereo audio headset. Many other configurations are possible. YouTube video clips of training during SBIR phase I evaluation study on The vOICe by MetaModal LLC Cost indication: depending on your personal trade-offs between speed, size, weight, battery life and your budget, cost of a new netbook or tablet may land in the $300 to $800 range. This will usually be the biggest cost factor, to which you should for a wearable setup add, say, a $50 backpack, $50 USB camera glasses and $25 stereo headphones or ear buds. A complete netbook-based mobile setup may thus be assembled under $500! Note: the netbook, tablet or notebook PC can obviously be used for many purposes other than wearing The vOICe, so its cost should not be attributed solely to The vOICe.

+ Show more (expand this page content)

The vOICe for Windows software fully integrates visual input from your digital PC camera (webcam, or video sunglasses with video capture device) with auditory output to your stereo headphones, while taking advantage of much improved synthetic vision processing through real-time audio and video processing algorithms, optimized for use on modern notebook PC's. Video frames are scanned from left to right. Object elevation simply becomes pitch: the higher the object, the higher the pitch. The brighter an object, the louder its sound. With the new hifi stereo output, objects on the left or right also sound on the left or right, respectively.

You can mute The vOICe for Windows at any moment with one button-press, e.g. to better hear traffic noise for safety, or just when you are talking to someone, or when you prefer silence, or ambient sounds and sound shadows. The same button immediately reactivates The vOICe for Windows when pressed another time.

If you own a modern PC, then probably only the PC camera forms a new addition to your hardware, since the other items are nowadays already quite common with multimedia PC's. Suitable PC cameras nowadays sell at less than $50, although one can of course always get better quality and more convenience at a higher price. If you already have access to a notebook PC and want to find out if The vOICe technology really is for you, you can start out very cheap by creating your own home-made setup, with a regular webcam mounted on a hat or helmet to make for an affordable "wearable webcam". Make sure that the PC camera of your choice fits your computer configuration and check that you have an appropriate video driver installed as described in your PC camera manual, or The vOICe for Windows (or any other PC camera software for that matter) will not recognize your PC camera. Ask your computer dealer to advise and help you, if necessary.

If you experience problems, please read the Frequently Asked Questions page to check if you can find the solution there. If you find any bugs or glitches in The vOICe for Windows, please report. However, positive reports are also very welcome! Good results cannot be guaranteed, but much effort will be spent to make improvements wherever possible, based on your feedback. Have fun exploring, but for your own safety make sure you get adequate O&M training support in installing and properly using The vOICe for Windows in orientation and mobility situations! Ask your O&M service provider and your computer dealer what they can do for you.

For detailed usage information, and a description of the advanced options, check out the

Advanced options

In the longer run, The vOICe for Windows may find applications in outdoor orientation and mobility, provided that you can indeed learn to master the interpretation of the complex sounds that correspond to complex visual scenes. With sufficient training, the risk of veering in crossing the street may be reduced because you can now actually see/hear the buildings on the other side of the road. Orientation and wayfinding may become easier when you preview and perceive the buildings, landmarks and other objects that are within visual range. After all, that's the way the sighted do it. Furthermore, it may make it easier to build a more complete mental map of your environment and reverse routes. You can see/hear the bright warning colors of textured tiles long before touching upon them, and thus more easily locate them and also see/hear and anticipate changes in track course and curbs well before the cane makes you notice all that - if at all. Moving around with The vOICe setup can give you a sense of distance, direction, elevation, size and visual texture, not just for a single object, but for the multiple objects and landmarks that make up your environment. Still, for safety reasons you should always keep listening for the natural environmental sounds of trees, traffic, and so on. Also, the cane will remain very important for ground-level protection, because auditory vision has its limitations. However, The vOICe for Windows does offer new and extended modes of perception and independence to the totally blind, going way beyond what you can do with the long cane and guide dog.

What does it not do? Soundscapes from The vOICe are not suitable for efficiently reading print; OCR with speech synthesis generally works much better for such purposes. Also, The vOICe for Windows does not include the image and soundscape editing, spectrographic analysis and other educational and ``fun'' features offered to the sighted by The vOICe Java applet/application (JavOICe), because The vOICe for Windows targets usage in the most immersive environment of all: reality! For this reason, The vOICe for Windows is of limited use without a PC camera or equivalent video capture device. Without a camera, you will hear a spoken error message saying ``I'm sorry, The vOICe cannot find your webcam,'' after which the demonstration mode is started. Still, you can subsequently make use of the built-in functions that do not require a camera, such as the exercise mode, the math plot mode, the GUI sonification options and the image file import.

Finally, it is, and always should be, your choice whether you want to use something like The vOICe, or not. Yet it is your option now, a new one, and an extremely powerful one. In fact, it is so powerful that it will take you a very long time to fully exploit all the visual things that you can now see/hear.

Mastering it all will require a lot of effort and practice, perhaps comparable to learning a foreign language, or perhaps comparable to learning to play the piano.

Right now, no one knows where it ends or what can be achieved. One experienced late-blind user of The vOICe put it like this: "Most people want that magic bullet giving instant sight. While this does occur over time, it is a developed thing." Indeed the first impressions with seeing with sound may be similar to the reported confusing experiences of the rare blind people who got their biological eyesight partially restored by surgery, as happened to Shirl Jennings, whose life story was chronicled by Oliver Sacks in his book "An Anthropologist on Mars" and inspired the movie "At First Sight".

Seeing with sound will be a major challenge too: vision itself happens to be very complicated - but The vOICe mapping is not! Moreover, if offers you uncensored access to any visual information, whereas a verbal audio description of a scene by a sighted person, however useful and accessible, inevitably always contains an element of censorship: even with the best of intentions, a concise verbal description of a scene by another person can capture only a tiny selection of what is visible at any time. That will always remain a fundamental limitation of for instance closed captioning of digital video and movies. By contrast, The vOICe ``speaks'' in an extremely dense form all the visual items, without personal preference, without interpretation or bias, leaving it up to you to focus your attention and try and interpret the things that interest you.

---

Many improvements are the result of blind people who worked with and gave constructive feedback on earlier implementations of The vOICe. Special thanks go to David, Pat and Pranav for their pioneering work on the user side. Please feel free to report your experiences with The vOICe for Windows, such that your feedback and suggestions can be used to develop further improvements and to help other users! Blind users of The vOICe for Windows are most welcome to join the seeingwithsound user group (mailing list) for sharing experiences and ideas.

License restrictions: You may freely and without cost apply The vOICe for Windows for personal or academic use. You will not charge a fee or request donations for The vOICe for Windows, nor distribute or include The vOICe for Windows in or with commercial products, nor modify or reverse engineer The vOICe for Windows. You will not use The vOICe for Windows for commercial purposes.

Disclaimer: THIS PRODUCT (The vOICe for Windows, also named The vOICe Learning Edition) IS PROVIDED AS IS WITHOUT ANY WARRANTY OF ANY KIND. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, PETER B.L. MEIJER FURTHER DISCLAIMS ALL WARRANTIES, INCLUDING WITHOUT LIMITATION ANY IMPLIED OR STATED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. THE ENTIRE RISK ARISING OUT OF THE USE OR PERFORMANCE OF THIS PRODUCT AND DOCUMENTATION REMAINS WITH RECIPIENT. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL PETER B.L. MEIJER OR HIS SUPPLIERS BE LIABLE FOR ANY CONSEQUENTIAL, INCIDENTAL, DIRECT, INDIRECT, SPECIAL, PUNITIVE, RECURSIVE, OR OTHER DAMAGES WHATSOEVER (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF BUSINESS PROFITS, BUSINESS INTERRUPTION, LOSS OF BUSINESS INFORMATION, PERSONAL INJURY, DISRUPTION OF FAMILY LIFE, OR OTHER PECUNIARY LOSS) ARISING OUT OF THIS AGREEMENT OR THE USE OF OR INABILITY TO USE THE PRODUCT, EVEN IF PETER B.L. MEIJER HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. BECAUSE SOME STATES/JURISDICTIONS DO NOT ALLOW THE EXCLUSION OR LIMITATION OF LIABILITY FOR CONSEQUENTIAL OR INCIDENTAL DAMAGES, THE ABOVE LIMITATION MAY NOT APPLY TO THE RECIPIENT. LET IT BE KNOWN THAT THE USER OF THIS PROGRAM HAS PAID PETER B.L. MEIJER THE SUM TOTAL OF $0.00 FOR THE USE OF THIS PROGRAM. ANY DAMAGES AWARDED SHALL NOT BE IN EXCESS OF SAID AMOUNT.

Supplemental information associated with disclaimer:

Concerning privacy of the observed: The vOICe for Windows by default (i.e. in normal use) does not store or transmit any video or snapshots; it converts live video frames to sound and next discards both the video data and the generated sound data. The vOICe for Windows is intended for educational uses, while its effectiveness or benefit has not been independently demonstrated and formally validated. It is not intended to diagnose, treat, cure, or prevent any disease or condition. It does not replace long cane and/or guide dog. Depending on national law and regulations, usage of The vOICe technology may be subject to approval by an institutional review board. Also, as with any audio-based system, permanent hearing loss may result when sounds are played at high volume (> 80 dB, OSHA) for prolonged periods of time, while environmental sounds can be partially masked even at a modest volume. Also note that this is still a research project, while the software may contain bugs, so there has to be a disclaimer with respect to any damages or harm that use of this educational software might cause through inappropriate use, just like you shouldn't attempt to dry your cat in the microwave oven. Never rely on The vOICe for your safety. Getting professional rehabilitation support, including adequate O&M training support for mobile orientation and mobility use, is considered extremely important when moving beyond mere educational uses, but even with that, 100% safety can never be guaranteed. Finally, some of the third-party product information may prove inaccurate, incomplete or outdated as it reflects a best effort to describe selected developments in a complex and highly dynamic market where adequate and reliable documentation is often missing or hard to track down. See also the privacy policy and disclaimer page.