Synesthesia – Extension of Perception for the Blind, Visually Impaired and Sighted

Synesthesia was an exhibition and workshop series where artists and organizations from Budapest, Bratislava, Prague, and Lublin collaborated to engage people with visual impairments in discussions about sound perception in ordinary experiences. In one interview, Peter, a visually impaired mathematician from Bratislava, explained how he associated the sound of a bubbling stream in nature, with the sound frequencies of speech synthesis; an aid tool software he uses every day. The sound collage he created for the project elaborated upon the conversation about his synesthetic experience of hearing voices of water. His auditory reconstruction of his experience was exhibited alongside the interview extract.

[Peter]

I love the sound of running water. I mean water out in nature. I often sleep outdoors, preferably beside a bubbling stream, because that's something I enjoy immensely. This is a very relaxing experience, though I can hardly explain why. Once, as I was falling asleep, I experienced a strange feeling: I began to understand the sound of water. Blind people working with computers normally use a speech synthesizer. Experienced text-to-speech users are able to understand synthetic speech at a relatively fast pace. For example, 500/600 words per minute is a comfortable speed for me.

[JK,AB]

What is speech synthesis?

[Peter]

Speech synthesis is the computer-generated simulation of human speech. I use this tool when I need to understand what is displayed on the computer screen. This is enabled by a synthesiser program designed to convert any written text on the screen into a spoken voice.

[JK,AB]

What did you mean by saying that you started to understand the sound of water?

"SZINESZTÉZIA," 2B Gallery Budapest, 2012.

"SYNESTEZJA," Galeria Labirynt Lublin, 2013.

[Peter]

That was a sort of half-dream. When I was almost half asleep, I began to hear some words coming from the sound of the bubbling water. I cannot remember those words; they were in fact meaningless, they only sounded like words. I don't know whether it ever happens to you, sighted people, that certain sounds remind you of something. Synthetic speech is far from natural human speech; it sounds rather robotic, so you must get used to it. When you listen very carefully to certain sounds, for example to the sound of a trickling stream, you may hear something familiar in that sound, like monotonous tapping for instance, because the water may be flowing between some rocks, and with a bit of fantasy, or when one is half asleep, as all the sounds mix together, you may all of a sudden understand something in the running water or out of the blue hear a word.

[JK,AB]

And you’ve associated this with speech synthesis, haven’t you?

[Peter]

Yes, mainly because speech synthesis produces robotic speech. Hence, it is very similar to the noises you can hear in your environment. If you listen attentively, any sound may be compared to a voice produced by a speech synthesizer. For example, the sound of flowing water has much more in common with synthetic speech than with human speech. A human voice could hardly be identified in that sound.

[JK,AB]

What words did you hear specifically?

[Peter]

I can't remember. It was a long time ago, last summer.

[JK,AB]

András, just told me that he also experienced something similar; once he set his ringtone to a high-pitched whistle at the level of one kilohertz. He then began perceiving various sounds of that frequency, often thinking it was his phone ringing. Mostly it happened when he was in urban environments.

[Peter]

Yes, that’s it. I also think that speech synthesis is only applicable in a limited frequency band. This band is relatively narrow, because synthetic speech is produced by a computer program which is designed to generate speech in the most economical form, in terms of the amount of data. Hence, we have a more sensitive perception of that particular frequency band. The frequency range of sounds produced by a stream is very wide; it includes a great variety of sounds. Being trained to listen to certain sounds, our brain seems to give preference to sounds that are attractive to us or to sounds we are accustomed to hearing.

Peter, Judit Kontsek [JK], András Blazsek [AB], Bratislava, February 1, 2012. The interview was published in the Synesthesia/ Szinesztézia/ Synestezie/ Synestezja catalogue in 2013.