Category Archives: sensory substitution

Perceptions: part II.

It was raining. Safe from the downpour, I stood on the porch and enjoyed the weather. Fall is one of my top five favorite seasons, especially if I don’t have to be out in the rain. It was really coming down too, the air filled with the hissing roar of all that water smashing against all that ground. Against the roof, a more percussive pounding, while the bushes and trees in my yard played host to myriad tiny streams and rivulets cascading down their leaves and branches.
Continue reading Perceptions: part II.

This morning, sights, and sounds

You’d think, after a post like “Perceptions part I” that the next thing I wrote on the subject would be “Perceptions part II.” Those posts, including the one not written, are about sensory substitution and visual qualia, and my experiences over time. Today, I want to talk about today.

I pull open my bottom desk drawer, and pull out the battery. That’s about the size of a deck of cards, only a bit wider, a tiny bit thicker, and much much heavier. A USB cord is still plugged in, the rest of the cord wrapped around it. It only takes a second to unwind the cord, then the battery goes in my breast pocket. I’ll have to buy more shirts that have one. I just never considered the functional constraints of upper body clothing before.
Continue reading This morning, sights, and sounds

Perceptions part I.

It was somewhere between first and fifth grade, closer to the former than the latter. Hanging around in the classroom, I was half listening to a couple of the teachers talking. One of them complimented the other on the decorations.

“thankyou,” she replied, “I like to have something with color… you know, something cheerful for the students”

“Then what’s with the upside-down smoke?” I asked. A moment passed, both teachers giving me quizzical expressions that I could only partly make out. I waved vaguely toward the back wall, where some of those colorful decorations were displayed. I could make out some of it—elephant, cheerful sun, friendly looking small cloud . Lost in a blur, most of the rest of it seemed a meaningless jumble, at least, from where I usually sat during class, and from the front of the room, where the three of us stood at the moment.

“What smoke?” the question stumbled over itself as they spoke in an awkward unison.

Using one hand, I shaped my fingers to indicate the part of the decoration I was talking about, almost like grabbing the image. “Right there, and then it curves up…?” I moved my arm to trace the arc. I’d been wondering what that thing was supposed to be a picture of. It looked like dark smoke rising in a column from a fire, curving to the side as though being gently pushed by a breeze, only it was upside-down. The bulbous top of the smoke was on the bottom, and the base up above it. The smoke, if smoke it was, curved down instead of up.

“That’s a rainbow.”
Continue reading Perceptions part I.

On the practical application of artificial vision

My robo-eye is sitting in my desk at the moment. It’s a pair of smart glasses, with a camera built in. I put these things on and run the vOICe, an app that changes what the camera sees into sound. With practice, blind folk like me can use such a setup to help navigate the world. As it turned out, I found they could help me do that within a couple of days.
Continue reading On the practical application of artificial vision

Accidentally reading without sight

Once I had the vOICe installed on my smart glasses, I put them on and played with it. The vOICe is an app that takes images from a camera, and turns them into sound. If one is blind, which I am, it can provide a way to send visual information through the sense of hearing. It takes time and practice to learn. For more information, or to download a copy to play with, you can check out the seeing with sound website.
Continue reading Accidentally reading without sight

The first thing I learned, was to turn on the light

I don’t use light in my house. I can’t see it, sometimes it doesn’t occur to me until someone comes over and asks for the stuff. Then I can tell an old joke that still gets a chuckle one time in ten, and go turn on the lamp I keep for just such occasions.

I put on this pair of smart glasses. They’re sort of like having a smart phone strapped to your face. I’m told the display is rather nifty; though of the two people who have checked it out, one complained of eyestrain, stinging in his eyes from the very close, slightly too bright light; and the other wears prescription glasses, so the display ended up rather blurry for him.
Continue reading The first thing I learned, was to turn on the light

Smart glasses, blindness, and shopping

Yesterday, I got email from the relevant companies. The smart glasses I ordered are on the way, actually shipped this time. This marks the end of a long, and quite Ludacris saga. I wondered just how long it really has been, so I took a look in the journal where I keep track of this mission. It’s taken more than five years, almost six.
Continue reading Smart glasses, blindness, and shopping

Identifying objects with the vOICe

As I said in the previous post, trying to figure out which object was which by tossing them wasn’t quite working out. Later that day, I had my assistant assist. He’d flip a coin, and then set one or the other of the two clay shapes in front of me, and I would say which one it was. It was either a small cube, or a small tetrahedron. I made some mistakes at first, but by the time the session was done, the difference between the way one sounded and the other was obvious. The tetrahedron has a sound that almost comes in two parts, as I hear it angle up and back down. I could hear the sharp peak, the roughly 60-degree angle at the top.

I’m told by @seeingwithsound, that a tetrahedron and cube are possible for this exercise, but difficult, and that a cube and sphere would be easier. I might squish the tetrahedron into a sphere and give that a try. It’s easy now when it’s one object at a time, but it might become more confusing when I’m trying to figure out which one is which and they are side by side. Maybe I’ll try a sphere today, and go back to the tetrahedron the next.

After another couple of days, using sphere or tetrahedron with the cube, one object at a time, we’ll try having them next to one another. I’ll have him flip a coin to decide which shape goes on the left and which one on the right. After that, I’ll have to try having them in line, so that one is behind and a bit further away than the other one. Note that I’ll have to stand up for that part—otherwise, the object that’s further away can be blocked from view by the object in the foreground. Sighted readers and those who lost their vision later in life, like me, might find that obvious, but those who have never seen may not know that can happen.

It’s 3:13 in the morning, so it won’t happen until considerably later in the day.

First shot sorting with the vOICe

My clay is setting. It’s a polymer clay, so first you squish it without mercy until it gets soft enough to work. Next you squish it carefully and oh so gently into the shape you want. Then you give it a while until it gets a bit harder. After that, you can toss it around without it quickly turning into a featureless blob.

Today, to change it up a bit, I’m going to do the sorting exercise. I made a little cube, and a tetrahedron—a four-sided figure like a pyramid with a three-side base, or a four-side die. That gives me two differently shaped, light-colored targets to set against a nice dark background, and take a gander at.

The goal is to learn to tell which one is which by listening to the vOICe instead of feeling them. You can checkout episode 19, if you want more info on the vOICe; or you can visit their website to dig a bit deeper, and even download a copy for free for your very own.

Well, that’s interesting. The tetrahedron sounds swoopier. Still, I’m having trouble when they happen to land next to one another. I think I’ll start by trying to figure out which shape I’m looking at, when it’s one shape at a time. Unfortunately, if I pick one, I know which one it is before I look at it. Well, this is the sort of thing I pay my assistant for.

Some days are better than others

I just finished this morning’s session with the vOICe. My performance was less effective than the last few sessions, so far as reach and grab, or reach and get close. I only had visual qualia a few times. More than once, I “saw” it, and then reached for the wrong spot, only to find that it was where I’d “seen” it. But the visual qualia came and went so quickly that it was difficult to localize the target. What’s more, when I started, I kept seeing several targets on different spots of the bed, ghost images.

I could avoid posting about sessions that don’t go well, but somewhere out there, someone else who is training with the vOICe, or considering using it needs to know that some days will go better than others. What’s more, gaining facility could take months, or years.

If you consider neural network entrainment, one could assume that today was about testing the model that my brain has built up so far.

I’m using the vOICe and my own experience as part of the overall research efforts, attempting to find ways to improve my ability to use my own mind and brain in a more optimal fashion. However, because I’m trying a number of different things, if I do suddenly find myself learning at a considerably faster rate than the average, it will be difficult to tell which thing or combination of things caused that, or if I just happen to luck out and be unusually good at learning this task, just because of the way I’m wired, regardless of my approach. I could also end up being slower than most, but still be better than I would have been without the experiments. Or perhaps the experiments might be counter-productive. There are so many confounding factors.

Meanwhile, I find that my instinct is to inhibit my emotional reaction, to try and hold down my level of frustration. This is something I do more or less automatically. Yet those with savant syndrome often have trouble regulating their emotions. That’s also true when one considers the tortured artist, mad scientist, or even children. Perhaps, the neural chemical reaction associated with frustration helps to refine the neural patterns that are being build up as my brain models the process. Perhaps I would learn better if I did get upset, so long as the frustration doesn’t interfere with my motivation and push me away from continuing to practice. All respect to Shawn Achor and the positive psychology folk with their “happiness advantage,” but just because something feels unpleasant, doesn’t mean that what’s happening is “bad.”

Oh, for a budget, and willing participants.

Meanwhile, more Tetris effect. In fact, as soon as I was done and sat down to turn on my computer and write this, I started to hear the sounds of the vOICe, so at least some automatic processing is still going on. Here’s one place where some insight strikes me as useful. I could fight it—try and push the back of the head noises completely out of my head. Instead, on the theory that more processing is better, I do my best to let it run.