A few days ago, I got an idea that it would be funny to cross modals. That is, convert visual information to audio, and vice versa, the idea for vision to audio is simple and I started working on some of the required sub-systems. Until a friend of mine, after informing him of the idea, informed me that such a system is already devoped.
The most interisting part is itthat the transormation function is working exactly as I would have done it myself, and it is reported to make it possible to make blind people see! coool.
Vision->audio explenation.
The scientific consideration is possible to find at http://www.artificialvision.com/: ( sample rate, human sensory bandwidth problems etc.).
But a simple overview is given on the figures at: (source), I am apparetly not allowed to show the image here, so you have to navigate to the adress to see the image.
To make the sound experince more smooth, I am considering if adding a gausion over the entire image would help, the referenced system is not trying to smoothin the audio representation from one image to another, but considering the very slow framerate the system seems targeted at ( around 0.5-1 sec for each image playback) , it might be a bad idea to throw away information, there seems to be a problem regaring the bandwidth of the ears, the framerates I would aim for would probably only give a headache.
anway, if you are interested, both the source, and a working windows application can be downloaded from http://www.artificialvision.com/.
No comments:
Post a Comment