USB Camera Glasses for the Blind

« (old camera glasses page, discussing some older/other glasses types)

This web page is no longer actively maintained and may therefore be partially obsolete.

It is now possible to have a head-mounted setup with USB camera glasses and wide angle lens for under $60! Complete setup with netbook PC well under $500!

Featured on Swiss television, May 2012: "Blinde können mit den Ohren 'sehen'", in the news program "10vor10". Video  Mit den Ohren sehen.
Featured on German television, November 2011: "Mit den Ohren sehen - die Hörbrille", in the popular science program "W wie Wissen".  Video.
Featured by New Scientist, August 2010:  "Sensory hijack: rewiring brains to see with sound". The New Scientist video clip is available on  YouTube.

An exciting development for blind users of The vOICe is the availability on eBay of  camera glasses with integrated wide-angle lens (USB device ID 05e1:0b01). The camera glasses depicted on the right are specified as offering a 170 degree field of view. These glasses can be bought for $50 - $70 including shipping (at the time of this writing), and although "PC camera" (webcam) support is often not advertised and drivers may not be included in the product package, we found that they do function as a USB webcam in combination with the Syntek STK03N webcam driver described below. This is also described in small-print manual that comes with the glasses.

After driver installation, a simple long-press on the power button on the left leg of the glasses, while connected to the PC, brings the glasses into webcam mode. Convenient for blind users is that the glasses emit a brief and weak high-pitched beep when they switch to webcam mode through long-pressing the power button. This beep runs several seconds ahead of The vOICe detecting that a USB device got connected.

Beware that some Windows tablets may have a single special USB port (a USB OTG port) that is also used for charging, in which case you would need to use a so-called  USB OTG cable for connecting peripherals such as a USB webcam or USB camera glasses. Moreover, there are initial user reports that indicate that the USB OTG cable solution does not work with the camera glasses discussed on this web page. Case in point is the Dell Venue 8 Pro that reportedly would not work.

 
Driver installation

The glasses typically require a Syntek STK03N webcam driver, e.g.  "STK03N.exe" ( mirror). After installation of the STK03N driver there should be a folder

In case Windows pops up its New Hardware Wizard when activating the camera for the first time through a long-press on the power button, manually select the option not let it search for software, click Next, and then allow it to install the software automatically (default), while giving the requested permissions during installation of what is named "Standard_Camera". This procedure was tested on Windows XP and Windows 7 64-bit.

You may find that the STK03N driver launches an "Image Download Utility" pop-up window. This window normally disappears automatically after 15 seconds (unless you have a micro SD card with media files in the glasses). You can keep the "Image Download Utility" window from appearing altogether by running msconfig, unchecking the "Syntek Camera" item in the Startup tab, pressing OK, and finally rebooting. Alternatively, you can delete the shortcut "STK03N PNP Monitor" (pointing to executable STK03NM.exe) from the Windows StartUp folder, and reboot or kill the process STK03NM.exe with Windows Task Manager.

 
Foveal mapping

To offer a better trade-off between visual detail and peripheral sight, The vOICe offers a foveal mapping option (toggled by Shift Alt F4) that enlarges the central part of the view. In combination with a wide-angle lens this preserves the peripheral view for detecting obstacles and events in the visual periphery, while offering a detailed central view.

The vOICe's foveal mapping has the additional property that horizontal and vertical straight lines remain perfectly straight, thus limiting the inevitable distortion associated with a foveal mapping. Unlike with for instance barrel distortion, an upright rectangle remains a rectangle with straight edges no matter if the rectangle appears in the foveal area, in the periphery, or in both. A settings menu (toggled by Shift Alt 4) enables correction for a camera view that is often physically somewhat misaligned with the frame of the glasses. It lets you adjust the effective viewing direction through software!

A video clip with Pranav Lal, a blind user of The vOICe in India wearing the wide-angle camera glasses, was broadcast by CNN-IBN in November 2012. To give a further impression, a video recording with soundscapes of a walk in the garden was made using camera glasses with a 170° fish-eye lens, and using The vOICe's default video resolution setting of 176 × 144 pixels. This same five-minute video clip is also available - with closed captioning - on  YouTube.

 
Google Android? Linux?

Although there exists a version of The vOICe for Android that runs on self-contained smart glasses (augmented reality glasses), there seem to be no camera glasses on the market yet that can interface with regular Android devices (it is possible to run The vOICe for Android with the USB camera glasses discussed on this web page by running it under the  BlueStacks App Player on a Windows PC). Some Android devices, notably TV sticks such as the Rikomagic MK802 IIIS and Rikomagic MK802 IV, include an UVC (USB Video Class, AKA Universal Video Class) driver and thereby support  UVC compliant cameras. UVC compliant camera glasses are still quite rare (an example of UVC compliant camera glasses is the Technaxx TX-25). So for now, camera glasses are best used with The vOICe for Windows for devices that run Microsoft Windows. Android phones would have to support having an external camera, would have to support the so-called USB host feature, and one would additionally need a Linux driver for the camera glasses, much like one would for using a Linux PC with the camera glasses. In many cases there only exist proprietary Windows drivers for USB camera glasses and for USB webcams in general. However, in case you do have Linux drivers for camera glasses, you can use the fully functional OpenCV-based sample source code hificode_OpenCV.cpp discussed on the Image to Sound Mapping web page as the starting point for a Linux version of The vOICe, e.g. for  Raspberry Pi based setups.

 
Known issues and their solutions

Camera glasses products are often poorly documented, while some of them do not support use as a webcam. If you have information that may help other buyers avoid buying glasses that do not properly work as a webcam, please report. Note that some camera glasses come with USB cables that were found to be critical in timing or power requirements: if your new glasses seem to fail, first also try other computers, preferably with a USB 3.0 port, before concluding that the camera glasses or their cables are broken. Also note that although high resolution recording may be advertised with the glasses, the highest available resolution in webcam mode is typically VGA (640 × 480).

An issue remains the fragility of the mini-USB connector of the glasses, which reportedly starts failing already after a short period of mobile use, possibly due to poor internal soldering connections as the connector can freely wiggle, albeit slightly, with respect to the left leg of the glasses. Therefore it is strongly advised to completely fix the plugged-in mini-USB connection with the glasses using a hardening filler epoxy for maximum mechanical stability and longevity of the electronic connections. To avoid accidentally getting epoxy inside the nearby microSD card slot, it is advisable to temporarily cover this slot with tape while applying the epoxy. Power dissipation of the glasses is less than 0.4 W, such that having it powered by a netbook PC should not dramatically reduce battery life of the netbook PC. Some camera glasses come with a 4-pin 2.5mm headphone-jack connector instead of a mini-USB connector. Such connectors may prove mechanically more durable than mini-USB.

In case the glasses keep slipping off your nose, especially when looking down, you can make use of adjustable eyewear retainers such as the so-called  "croakies" attached to the legs of your glasses.

In rare situations, for instance after a system crash, it can happen that the camera glasses are no longer recognized by the computer at all - not even in their default memory card mode. This need not mean that the glasses are broken, but that a reset of the glasses is needed to solve the problem. On the bottom side of the left leg, right behind the mini-USB connector, there is a small gap containing the reset button. The button switch gives a noticeable tactile click when depressed with a thin object (e.g. a needle). Be careful not to push too hard and risk damaging the switch.

``My vision is getting more detailed now with the new glasses and I am actually choosing patterns of things because I like them, I can go shopping and say "I like the pattern of that rug so I will have that one", or clothes, etc. it is very interesting to see things in such detail. I did not know what quite a lot of objects looked like and could not visualise them. I seem to like patterns of Moroccan tiles because they have a repetitive sound, rather like the chorus of a song.'' CC, female blind user of The vOICe, March 5, 2010.

First run with my new MINI DV D008 USB camera sunglasses (Technology Esoterica blog) Pranav Lal, early blind user of The vOICe, April 24, 2010.

A Picture from New Year's Eve (The Darkness Escapee blog) ``I am trying to learn about 3D shapes and finding it more difficult than I could have imagined. Starting on New Year's Eve, I've examined this cube from many angles in order to construct a mental 3D representation.'' Amanda, early blind user of The vOICe, January 7, 2012.

Whether you are blind or not, beware of possible legal and security implications of buying and wearing a (concealed) camera in your country. The relevant laws differ per country.

 
Learn to see

A good starting point for learning to see with sound using The vOICe is the table-top grasping exercise, but make sure to also check out The vOICe Training Manual.

 

Sighted volunteer users are most welcome in this open-ended project! In order to determine how visual the experience of seeing-with-sound can become over time, input from sighted volunteer users is highly appreciated! Perhaps you can help? If you already own a netbook PC and headphones then you only need to add camera glasses (see above) to actively participate and contribute to our understanding of what it takes and means to see with sound. What you do may one day make a real difference for millions of blind people around the world. For sighted users it is recommended to wear the USB camera glasses with The vOICe for Windows's foveal view setting, supplemented now with opaque clip-on sunglasses on the camera glasses (see photograph). Regular clip-on sunglasses can be made completely opaque by covering them with for instance thick black tape. At a local optician you may buy clip-on sunglasses that have not yet been cut to fit a specific pair of glasses, such that a wide field of view is blocked by making the clip-on sunglasses opaque. The resulting glasses setup blocks most natural eyesight (all central vision and most peripheral vision) but not the camera lens and its central and peripheral view via The vOICe. Moreover, the remaining peripheral eyesight lets you move around without stress in a safe (home) environment, such that you can really focus your attention on the sounds of The vOICe for most of the view, and perform visual training tasks using only The vOICe. Thus you can train yourself to reach spot-on for things like door handles, DUPLO bricks or any other objects in the camera view, and locate doorways and other indoor "landmarks" while walking around. You can study the extent to which the central soundscape view over the course of weeks starts to blend in with your peripheral eyesight, and how your visualization of the environment gets strengthened and "locked" to the soundscape content. You may find that at some point you hardly notice the sounds any more but rather perceive their visual content. In addition, there are some indications that blinking your eyes at a high rate may support soundscape visualization. Please wear The vOICe and practise for a minimum of 20-30 minutes a day and for a period of a few weeks. You are welcome to report publicly (e.g., blog) or privately about your experiences. You may also report to the seeingwithsound user group (mailing list). Thank you!