Smart glasses, blindness, and shopping



Yesterday, I got email from the relevant companies. The smart glasses I ordered are on the way, actually shipped this time. This marks the end of a long, and quite Ludacris saga. I wondered just how long it really has been, so I took a look in the journal where I keep track of this mission. It’s taken more than five years, almost six.

Back in 2012, while researching for an experiment with artificial neural networks, I ran across a concept called “sensory substitution.” Pioneered by Paul Bach-y-Rita in the 1960/s, the idea is to translate information normally experience through one sense into another one. With time and practice, the information gets interpreted by the brain, and can help partially replace whatever sense, such as sight in my case, might be damaged or even completely missing.

At first, I was frustrated. The early experiments had shown quite a bit of promise, but it seemed as though nobody had bothered to make anything to use the idea, probably because it was working. There had been one or two projects, but they had either never been followed through on, or whatever company had made the product had gone out of business. For a time, I thought I might even have to get into engineering prototypes—a problem, as I have no clue how to go about that.

Eventually, I found a couple of products that were on the market. One was a tongue stimulator. It takes information from a camera, and turns it into electrical stimulation on the tongue. The other approach is the vOICe, which takes images from a video camera, and turns them into sound. The former is expensive, and made by only one little company halfway round the world from my neck of the woods. The latter is free, and can be downloaded to any computer that has access to the internet. I decided to download the software, and started to play with it. If you’d like some idea of how it works and sounds, just press play on the podcast player above.

After spending some time playing with the vOICe on my computer, I was convinced that it was the right way to go. The software is just software, but it can work with nice standard off the shelf equipment. That brought me to the point where I needed to get the hardware I’d need to take the software, and my wittle self, out into the big bad world. That’s when things got strangely complicated and frustrating.

At first, I tried using a laptop and USB camera. I figured to stick the camera to the bill of my cap, and experiment with it for a while. Sounds simple, but every camera I bought turned out to be shaped just wrong, and none of them would stay on the cap. At least they weren’t that pricy, so I wasn’t out too much money. Even though it didn’t work out, I did manage to take the setup out into the field for a few brief trips. That was just enough experience to wet my appetite.

I recall being at a friend’s house. I had a camera on my cap, earbuds in my ears, a laptop on my back, and wires running everywhere. Before that particular setup fell apart, I’d chanced to tilt by head up. I expected to hear a featureless wash of sound, as the camera was pointing up at the sky. Instead, I heard something up there, something with texture and detail and depth. It took me a while to realize I was looking at clouds. My eyes stung slightly with unshed tears of joy. It was the first time I’d noticed clouds in years; I’d almost forgotten they could be up there.

The next thing I tried was to order a pair of camera glasses. These were just a camera set in some dinky looking plastic glasses that I could connect to my laptop with a not quite standard cord. Smart glasses hadn’t happened quite yet. the only such product on the market was Google Glass, and they had a four-figure price tag. What’s more, at the time, you had to go through an application process—Google wasn’t selling their super-duper new product to just any old so and so.

The camera glasses were much easier to work with, for the all of twenty minutes before the first pair broke. I tried to order another pair, and they got held up in customs, of all things. I finally got a replacement pair, and despite attempting to use epoxy to keep them from breaking, they broke in about ten minutes. That time, I lost a couple hundred bucks.

I tried to find a different product. Months slipped by as I begged rides from friends and family, went from store to store, getting reactions that ranged from baffled to faintly hostile. A couple of times, I heard about some little shop that would have camera glasses, only to find that the place had gone out of business a scant few weeks just before I needed them.

Much much later, I tried again. By this time, smart glasses had entered the market. This included an inexpensive model that was known to work well with the vOICe. I tried to order a pair, but for reasons that were never explained, the shipment was canceled. That time, all I lost was time. A few months later, and I tried yet again. This time, every last site I went to was suddenly difficult or impossible to use.

As a blind computer user, I have software on my system that reads the text from my computer screen in a synthesized voice. Mostly, the world and its wide web work well enough with the software, called a screen reader. Every now and again, some site or another doesn’t play well with others. I’ve an assistant who I pay to help take care of the thousand and one things I could use a set of working eyes for. I got him to sit in the camp chair I keep in my room for guests, right next to the lamp that is also just for guests, as I don’t especially need light. Even with his help, I couldn’t seem to just order the damn things. Finally, after a flurry of messages and some help from @seeingwithsound, the creator of the vOICe, I managed to order the damn things.

This time, according to the emails I’ve been getting, they’ve actually been shipped. They should show up sometime next month. Assuming customs doesn’t hold them up, or the boat they’re on doesn’t sink, or the silly things don’t break shortly after arrival, I should at last be able to start to use the vOICe.

Wish me luck, and by “luck,” I mean good luck.


2 thoughts on “Smart glasses, blindness, and shopping

  1. I liked your shared experience ref. those clouds . The vOICe has great subtleties in responding to textures, webs and ,of course, colours as an option.
    Your webcam will probably work at one or two inches from an object.
    You can easily discriminate different clothing fabrics, or indeed pictures of them.
    You can sort out different grocery items, such as jams and pickles in glass jars.
    You could with care sort out spilled medication in capsule form.
    Laundry…sort out those odd socks.
    The Smart glasses do not have an extra camera socket… but the PC and Raspberry-Pi systems do permit a handheld camera to be an option.
    I have put together a Finger-cam, which is a Raspberry-Pi spy cam in a cigar tube.
    This can be cabled down my sleeve, and popped out for peeking at things.
    Raspberry-vi dot com is the place where this sort of experimentation is going on.

    1. Hello John

      I just wanted to say thanks for the comment and for joining my mailing list. I hope you like being in on the start of things, as you are the very first person to join the list. It will take a while before regular messages come out. My show and the associated projects are not quite one year old. For updates, you can watch my twitter feed.

      Brad Barton (@lobbyandlab)
      https://twitter.com/lobbyandlab

      The smart glasses are on the way and should show up sometime this month. Keep an eye out for further updates on my adventures with sensory substitution, as well as the experiments with digital organisms, and of course, the podcast.

      Thanks for reaching out, and feel free to drop me a line about anything or nothing at all.

      Brad Barton
      http://thelabwithbrad.com

Leave a Reply