Microsoft helps blind artist add soundscapes to his landscapes

Microsoft has helped well-known legally bling Scottish landscape artist Keith Salmon add a sound element to his landscape paintings.

Salmon creates ethereal, moody abstracts of skylines and mountains inspired by the rugged highlands in his country and as his eyesight has diminished through diabetic retinopathy he has learnt how to smash paint into texture and scratch pastels for scribbles to evoke less a depiction of place and more an exquisite experience of the wild.

With Microsoft’s help he has debuted an installation which uses sound to enrich his art even more, with proxemic audio to interpret two-dimensional images.

Called “The Oregon Project,” it uses four Kinects, 15 overhead speakers and 54 soundtracks to produce an acoustic and spatial interpretation of three drawings Salmon did of the beautifully remote Hells Canyon area in Oregon.

In front of each drawing is a 3D space containing 18 soundtracks that can be played. The Kinects track movement to trigger different sounds, making the art lively and interactive for people with and without vision. From afar, observers can hear birdsong, rushing water and grass swishing in the wind — recorded in Hells Canyon by Salmon and his collaborators.

Moving closer activates digital tones matched to Salmon’s palette of blue, green, brown and ochre. Stand even closer and you can hear Salmon working in his studio as he scribbles pastels on paper, in an experience that parallels how a sighted person can see details when leaning in. The Kinects also track hand waves to change what you hear, helping observers become part of the piece and make their own acoustic mixes with movement. At the exhibit, adults triggered sounds primarily by walking, while kids jumped and waved their arms to produce a different blend of noise.

“As a way of moving from an outer to inner layer to the drawings, natural sounds to industrial sounds works really well,” Salmon says. “It has a wonderful, atmospheric, spacey, landscape-y feel that pulls the whole thing together.”

Many museums offer experiences for people with visual impairment, from detailed audio descriptions of art to special events where people can touch sculptures with gloved hands. But the researchers wanted to create a more personalized experience that was still communal, and something less directed and curated by docents.

“It would be great to think that in a few years, you might go to a national museum or art gallery and this technology would be there as part of the experience,” says Salmon.

The installation premiered in Seattle at the 9e2 exhibit of art and technology as a powerful new way for Salmon to create and for people with low vision to experience and enjoy visual art. The “Oregon Project” will be on display at the Tent Gallery at the Edinburgh College of Art in Scotland in April of 2017.

Read more about the work at Microsoft here.