Usage Notes for The vOICe for Android on Smart Glasses

Vadim Artsev († March 2021), a totally blind user of The vOICe vision glasses in Russia, won the  Neurothlon 2018 DBN competition that included an obstacle course and an object recognition task, beating not just one but two Argus II retinal implant recipients among the competitors! The sub-$200 VISION-800 smart glasses beat $150,000 retinal implants! Featured on Russian national television in November 2018, and more recently in the Russian article  Пионеры новых технологий: Вадим Арцев рассказал, как перестал быть незрячим. A YouTube  demo video for sighted people illustrates using basic geometrical shapes how The vOICe vision glasses convey vision through sound. The vOICe vision glasses will next compete head-on with the  Neuralink Blindsight brain implant, with many advantages: no surgery, higher acuity, minimal view distortion, larger field of view, lower cost, replaceable hardware, conveying textures and shading versus just a few edges to prevent seizures, globally available, working for people born blind (Neuralink's implant won't, no matter what Elon Musk says), unlimited daily use, ...

The vOICe for Android sensory substitution app for the blind nowadays runs on various types of smart glasses. Currently the usage notes given below are based on the sub-$200 VISION-800 smart glasses (e.g. try the  eBay search link, sorted by price + shipping, and optionally filtered for  eBay international sellers). Note that buying pages may appear inaccessible or cluttered, in which case you may need to ask a trusted sighted friend or relative for assistance, or in case of eBay contact the  eBay Accessibility Team. Much of what is written below should with minor changes be applicable to most other Android smart glasses as well. Just like other smart glasses, the VISION-800 smart glasses form a stand-alone device that independently runs Android apps inside the device. They are not a peripheral for a smartphone.

The goal of the usage notes is to minimize the need for sighted assistance. The VISION-800 smart glasses come with the Google TalkBack screen reader (part of the Android Accessibility Suite), Google text-to-speech and Google Play pre-installed. If you are blind then you probably need only one-time sighted assistance to activate the Google TalkBack screen reader, and should be able to do everything else independently. The self-contained glasses include two mono earbuds, one for each leg, that together give stereo.

• When your VISION-800 glasses first arrive, the camera lens on the front left side is likely still covered by a protective plastic film that should be carefully peeled off using your finger nail, or else you may later on get a somewhat blurry and hazy poor quality view once the plastic film starts degrading. Note that the thin protective film does not block your view and is not or barely visible even to the sighted, such that it may easily go unnoticed.

• The VISION-800 smart glasses run Android 4.4.2, and with this older version of Android there is no known way that a blind person can independently activate Google TalkBack. Therefore ask a sighted friend or relative to activate Google TalkBack for you in Settings | Accessibility. An external keyboard is needed to arrow down and press the OK button in the TalkBack activation dialog. The use of an external keyboard is further outlined in the next point.

• Connect a wired USB keyboard to the regular USB connector on the glasses, such that you will be able to navigate Android on the glasses without first having to pair a Bluetooth keyboard and face further accessibility issues in reading the pairing authorization code. Alternatively, you can use a wireless 2.4GHz (mini-)keyboard such as one of the Rii keyboards from  Riitek with the USB dongle inserted in the glasses. Note that you must disconnect the power cable in case you were charging the glasses, or else an external USB keyboard (or mouse) will not work. Moving among options and buttons is generally done with the arrow keys and activation is done with the Enter key, but sometimes only the left-right arrow keys seem to work and sometimes only the up-down arrow keys work, for no apparent reason: just try what works.

• If applicable, change the TalkBack keyboard shortcut settings from "Classic keymap" to "Default keymap", such that you have more shortcuts for navigating web pages, and specifically Google Play. Note that later versions of Google TalkBack may not have the option described here.

• Connect to your Wifi network using the password of your Wifi network.

• Log into your Google account ("Add account", which will usually mean your existing account). This may involve logging in a second time via a web page and providing a 6-digit authentication code obtained from the Google  Authenticator app on your smartphone. In many cases it is even easier than that, and the Authenticator app will only ask "Is it you trying to sign in?", and you press the "Yes" button to confirm. After that you can proceed on the glasses.

  Google Play

  Once connected to Google Play, the automatic update process will start for pre-installed apps, which may help fix regular pop-up error messages that you may have encountered up to this point e.g. with the Google or Google Play app. The automatic update process can take half an hour or so, depending on connection speed.

• Open the Google Play app, and use the Shift Alt arrow key combinations to bring focus to the search edit box in Google Play. Specifically, when on the App & games tab, press Shift Alt down arrow, Shift Alt left arrow, Shift Alt Enter to bring focus to and activate the search box, type "seeingwithsound" written as one word without spaces, and press Enter to find  The vOICe for Android app. From here on you only need the arrow keys and enter key to select and install The vOICe.

  By default The vOICe will launch automatically after a reboot (except when a screen reader is running) for blind accessibility out-of-the-box, but you can readily change this default behavior in The vOICe's own Options menu, via the entry Other settings. There you can also toggle the built-in cataract simulator on and off, a feature that is meant to let normally sighted users experience blindness in combination with The vOICe sensory substitution, whereas blind or low vision users may prefer the more detailed high contrast view that closely matches the soundscape content.

  APK file

  In case you do not have access to Google Play you can also get the latest version of The vOICe for Android directly as an APK file from www.seeingwithsound.com/phone/vOICeAndroid.apk and install it with  ADB using the command "adb install -r -g vOICeAndroid.apk", if necessary launch it from the PC with "adb shell am start -n vOICe.vOICe/.The_vOICe", and uninstall it with "adb uninstall vOICe.vOICe".

• Storage space on the VISION-800 glasses is limited, so you may wish to uninstall Chinese apps that come pre-installed on the device. With TalkBack you can typically identify these as apps that seem to have no name, only a size, because Chinese app names are not spoken by the English text-to-speech engine.

• When fully charged, the built-in 720 mAh 3.7V battery of the VISION-800 glasses lets you run The vOICe for over an hour. However, if you connect an external, say, 15000 mAh battery pack (power bank) then you can run The vOICe all day on a single charge. Start with the internal battery fully charged and it will remain at 100% until the external battery begins to yield.

  Congenitally blind user of The vOICe wearing VISION-800 smart glasses

• On the top side of the left leg of the glasses is a 3-way switch. This switch acts as a forward and backward navigation button when pushed forward or pulled backward, and as a mouse click or enter key when briefly pressed. A backward pull also doubles as the Android Back button, and a long backward pull will finally exit The vOICe if it was running. Now while The vOICe is running, forward (push) and backward (pull) moves let you navigate between software buttons for giving voice commands (when pressed) and for entering the Options menu (when pressed). When applying a quick "double-pull" or "double-push" (analogous to a double-tap or double-click), the soundscape scan time will halve or double along with the above navigation action, but you can subsequently compensate for the unintended navigation action by moving the 3-way switch in the opposite direction. This double action "interference" between scan rate selection and navigation serves to make best use of the very limited button set of the VISION-800 glasses. You will soon get used to it. For access to more specialized features you can always use an external keyboard.

• The power button of the VISION-800 glasses is located underneath the glasses body just to the left of your nose, and next to the SD card slot. The power button is really tiny and does not protrude much, such that initially you may have a hard time locating it even when scanning the surface with your finger nail.

• The head strap that comes with the VISION-800 glasses can help reduce the weight of the glasses on your nose (although the glasses weigh only 100 gram), or be used to hang the glasses around your neck when not in use. You can pull out the rear sections of the legs of the glasses by pressing the buttons that are located midway underneath the legs, and then plug in the strap connectors. Note that the orientation of these connectors matters: each connector has two protruding little pins with tiny "hooks", and those hooks should be pointing away from the detachment button, i.e., upward with respect to the glasses. Then the connector will snap in place with a sharp click that you do not get when inserting the connector with the wrong orientation.

• The VISION-800 glasses assume to be one-size-fits-all, but if you have a small head, or if The vOICe is used with young children, you may have to be a little creative with foam spacers to make the glasses fit well. The same applies to many other smart glasses.

ВИДЕНИЕ-800 Android смарт-очки  Tomtop. The vOICe для Android  Play Маркет. Учебное пособие по использованию The vOICe. Плейлист  YouTube. Научиться видеть с помощью звука:  Звуковое зрение vOICe vision (archived, no longer active).

The vOICe for Android is known to run also on smart glasses from  Vuzix ( Vuzix App Store),  Epson Moverio (Moverio Apps Market closed as of March 2023),  ThirdEye Gen ( ThirdEye App Store),  Optinvent and Osterhout Design Group (out of business as of late 2018), as well as on  Google Glass Enterprise Edition 2 (no longer supported as of September 2023). The vOICe for Android runs on Vuzix Blade smart glasses and even won the Vuzix Blade augmented reality glasses contest in the Personal category ( 2020 Vuzix press release), and also runs on the newer and more powerful Vuzix Shield smart glasses. Preliminary test reports indicate that The vOICe for Android also runs on the  RayNeo X2 glasses from TCL and on the  ARGO smartglasses from DigiLens that both have a camera in the bridge above the nose. Adaptations to the user interface may follow in future releases of The vOICe for Android after hands-on testing. Expand the text below for further details.

Other smart glasses that may be able to run The vOICe for Android (untested!) include the  Air 2 smartglasses from INMO. The vOICe for Android has not yet been prepared to detect these glasses as smartglasses as needed to automatically switch its user interface into smartglasses mode. Companies seeking to have their new smart glasses detected by The vOICe can contact the developer to provide the required android.os.Build parameters. Please note that sound quality of smart glasses is often mediocre at best, such that you may want to use them in combination with a Bluetooth stereo bone conduction headset such as the  Shokz (formerly AfterShokz) Aeropex. Good quality headphones make The vOICe soundscapes far less obtrusive.

Note that on smart glasses running Android 11 or later you may get a permissions dialog at app startup asking you to give permission to let The vOICe for Android display over other apps. This is an Android quirk, because this permission only serves to enable The vOICe to launch automatically after a reboot ("autostart"). This is a necessary feature for blind users if the smart glasses lack a screen reader, because in that case blind users cannot navigate the home screen on their own to manually locate and launch The vOICe app.

When using the cataract simulator as a normally sighted person, it is advisable to add opaque shades to the smart glasses in order to only see the degraded view for a realistic simulation of what it is like to have cataracts. Such shades are not needed for the VISION-800 glasses, because contrary to most other smart glasses they have no semi-transparent "see-through" display.

Congenitally blind user of The vOICe wearing Vuzix Blade smart glasses

Beware that it is currently unknown which smart glasses other than the VISION-800 glasses can make good use of Google's  TalkBack screen reader for blind accessibility. At the time of writing, Vuzix Blade 2 glasses come with Android 11 but do not include nor support the Google TalkBack screen reader, so you may need to install another screen reader such as the Jieshuo screen reader.

Controlling smart glasses with switches, scroll wheels or tiny touchpads can be laborious and error-prone, and sighted users may therefore while configuring smart glasses complement this by connecting the glasses to a PC via USB and using  scrcpy USB for more convenient access to the various settings on the glasses.

+ Show additional notes about smart glasses (expand text)

Unfortunately not: although the Quest headsets run Android apps, and although you can sideload and run The vOICe for Android on for instance the Meta Quest 2, it then only gives error messages about not being able to access the camera until it gives up after about half a minute. This access limitation is due to Facebook/Meta policies that block third-party Android apps (and web apps) from pixel-level access to the live (passthrough) camera view, because technically there is otherwise no reason why The vOICe could not be used on the Quest 2, Quest 3 or Quest Pro just like it can be used on an Android smartphone. It is hoped that Meta will someday lift this accessibility constraint, because the Quest headsets could then make for a nice training device for learning to see with sound.

Blind user of The vOICe wearing VISION-800 smart glasses

• The VISION-800 glasses are not designed to be fashionable, and they do not even look like ordinary glasses with lenses. They are like a thick bar in front of your eyes. As tastes vary, the "glasses" may give you a nerdy look that may or may not bother you in public places, or the glasses may actually make you look cool.

• The VISION-800 glasses do not support GPS, compass or accelerometer for location and orientation, and they do not contain a flashlight.

• During periods that other apps are updating in the background, The vOICe may be skipping soundscapes or otherwise show poor performance due to the extra CPU load of other processes. This should not be a problem when you are outside Wifi range, when Wifi is disabled, or when apps are done updating.

• If The vOICe's talking OCR feature does not work despite being enabled under Options | Other settings, try connecting the VISION-800 glasses to the Internet via Wifi, such that possibly outdated or missing Android libraries from Google, including Google Play Services, can be automatically updated in the background by the Android system. This might take up to a few hours. In case this still does not suffice then you may need to uninstall any apps that you do not really need in order to free up internal memory for those additional Android libraries from Google. Next, uninstall and reinstall The vOICe to reactivate any download triggers for Google's Android libraries. Note that these background downloads are not initiated by or under control of The vOICe app, but under control of the Android system.

• The VISION-800 glasses do not have a super-fast CPU, and especially at double speed scanning this can give small but noticeable hiccups between soundscapes. For smoothest results with soundscapes you can disable face detection, text detection and barcode detection under Options | Other settings. Face detection is already by default off on smart glasses, but text detection and barcode detection are by default on, despite the added CPU load, in a trade-off between functionality and overall performance.

• Text recognition by OCR will occasionally give false positive detection of text that is not really there, because shapes and textures in complex environmental images sometimes resemble print characters. If you wish you can disable OCR under Options | Other settings. Face detection may similarly sometimes report a face where there is none, and you can disable face detection in the same Other settings menu.

• The VISION-800 glasses give significant motion blur under low light conditions, due to longer exposure times and limited light sensitivity of the camera. It helps if you keep your head steady and move slowly. However, even when moving fast you can minimize motion blur by active tracking of visual items in your view: you slowly turn your head as needed to keep an item or pattern at a more or less fixed position in the camera view, and once that item is getting behind you you quickly turn your head back to another item that is more in front of you. You keep alternating these "head saccades" with the gradual and smooth visual tracking.

• With large changes in ambient light level it takes the camera driver a few frames to adjust its light amplification. This causes an apparent lag of a few seconds, but it is unrelated to The vOICe: it is a camera property. It is similar to the adjustment of a few seconds that sighted people need when walking from a sunny place into a dark room.

• Although the built-in battery should last over an hour on a single charge (at least when the glasses are relatively new), this can be less if you do not regularly fully discharge the battery, or if you keep charging the battery for too long (long-term over-charging). This is often referred to under the general term  memory effect. The best you can do in this case to regain lost battery capacity is to perform one or more deep charge/discharge cycles and avoid long-term trickle-charging of the glasses.

• The camera of the VISION-800 glasses is on the front side of the left leg. A position at the center would feel more natural, i.e. between the eyes or right above the nose. The slight offset to the left of the center can initially cause a bias in reaching and grasping. However, your brain should adapt to the offset after a few days. Moreover, it only affects accurate camera-hand coordination at close range. At distances beyond a few meters the offset is negligible and imperceptible (e.g. at a distance of 10 meters it gives a directional error of only 0.2 degrees).

• The VISION-800 glasses are rather fragile, so they must be treated with care to ensure a decent device lifetime. Sometimes, when pulling out a USB connector from the leg of the glasses, part of the electronics unit may also get pulled loose along with that USB plug. When carefully maneuvering and pushing the electronics back in place without breaking any of the internal wiring, all should be fine without damage or loss of function. Another weak spot is the nose piece. Although the rubber material is soft and flexible, it is held up by a thin hard plastic pin inside which easily breaks if the glasses are pushed down on the nose. So be careful not to do that. Once one of the two pins breaks, you will have to improvise a replacement soft part using for instance a piece of foam or cotton.

• Occasionally a full screen ad may pop up, e.g. when launching The vOICe. These ads are unrelated to The vOICe and block normal operation of The vOICe when they occur. You can readily exit the ads via a Back button press (backward pull of the 3-way switch). The origin of the ads appears be an app named "Shell" that is among VISION-800's pre-installed regular apps. You can uninstall "Shell" via device Settings | Apps | Shell. There will still remain a system component named "Shell" under the "All" tab that cannot (and probably should not even if it could) be uninstalled, but the full screen ads should no longer show up.

• The VISION-800 glasses run an old version of Android that no longer receives security patches. However, with Google 2-factor authentication, a router firewall for Wifi, and not browsing the web or reading e-mail with the glasses, the security risk is probably extremely small and limited in scope. However, absolute guarantees can never be given.

• Wired and wireless external keyboards stop functioning while charging the glasses, including when charging them from an external battery for prolonged usage. This can be a reason to use a Bluetooth keyboard such as the Rii RT-MWK02+ Bluetooth mini keyboard. Keep in mind that although you can first use a USB keyboard up to the point of pairing the Bluetooth keyboard, the 6-digit PIN number must be entered on the Bluetooth keyboard and not on the USB keyboard, or else you get an error message like (for the Rii RT-MWK02+) "Couldn't pair with Bluetooth 3.0 Macro Keyboard because of an incorrect PIN or passkey". However, TalkBack cannot speak the keystrokes on the Bluetooth keyboard before it is paired, so you either have to type the PIN "blind", or get one-time sighted assistance with pairing.

  

• The included pair of mono earbuds with 2.5 mm mono audio connectors block the ears for environmental sounds and echolocation, and are not easily replaced by other more suitable wired stereo headsets (which usually have 3.5 mm stereo audio connectors). This may limit use of the smart glasses to a familiar and safe home environment, which is strongly advisable anyway while mastering The vOICe. However, the VISION-800 glasses support Bluetooth, so it is possible to pair them with Bluetooth bone conduction stereo headphones such as the Shokz Bluez and thereby avoid blocking of environmental sounds by earbuds. If you want both your smartphone and the VISION-800 smart glasses to sound through the same headset, you may consider so-called multipoint Bluetooth headsets that can pair with more than one device at the same time (e.g. Aftershokz Trekz Air). This has not yet been tested though.

  In order to pair Shokz Bluetooth bone conduction headphones without sighted assistance, apply the following steps:

  1. Disconnect the VISION-800 glasses from any charger, and connect a keyboard.
  2. On the glasses go to Settings, and enable Bluetooth.
  3. On the headphones press and hold the power button for 5 seconds to enter pairing mode (discovery mode).
  4. On the glasses you should find the headphones listed in the Bluetooth dialog. Hit Alt+Shift+Enter on the name of your headphones.
  5. Follow the prompts. In case you are asked for a code, it is "0000".

  If the pairing does not work the first time, you can reset the headphones via the following steps:

  1. Start with the headphones turned off and cleared from your glasses' Bluetooth settings.
  2. Press the volume plus button for 5-7 seconds. This is also your power on/off button.
  3. Press both volume keys while you hold the multifunction function button down, so you hold 3 keys down at the same time, for 3-5 seconds. You'll hear beeps and/or feel vibrations.
  4. Turn the headphones off.
  5. Turn the headphones on again.

  Your headphones are now reset and can be re-paired to your glasses. If it still did not work then reboot your glasses.

• If the VISION-800 glasses start to spontaneously and repeatedly reboot, this need not mean that the glasses are broken. Chances are that its battery charge detector is confused about the battery state, for instance after many days of continuous trickle charging. The remedy is then to let the device run for as long as possible (including the spontaneous rebooting) in order to fully discharge its built-in battery. If the device appears turned off, turn it on a number of times to further discharge the battery. This procedure should reset the battery charge detector, after which the device may start working normally again after recharging the fully discharged battery.

  The VISION-800 glasses have a tiny reset hole on the inner side of the glasses, between the left hinge and the left eye piece. The reset button can be pressed by gently pushing a needle in the reset hole, which will turn off the device in case it was stuck in some way. However, it is not known if this helps with any of the problems discussed on this page.

• [Older versions of TalkBack only? The following problem has not been observed on more recent setups.] It seems that the 3D display mode toggled by a long-press on the 3-way switch on the left leg of the VISION-800 glasses generates an infinite series of screen mode change events that gets spoken by TalkBack as an alternating "portrait", "landscape", "portrait", "landscape", "portrait", "landscape", and so on and so forth. The events also seem to pile up faster than TalkBack can speak them, and only a reboot of the glasses gets rid of the stack of events. TalkBack speaking these repeating screen mode switches may even be a reason to disable TalkBack and rely on The vOICe launching automatically after a reboot. Most options within The vOICe can be reached and changed without TalkBack because The vOICe is self-voicing.