How to turn graphics into sound, and why?
Charts and infographics - these data formats are often not accessible to blind and visually impaired people. The developers of the American company SAS Institute have tried to solve this problem by creating a free browser-based plug-in Graphics Accelerator.
His task is to translate graphic information into sound. Today we talk about the development of SAS Institute and other cases where the sound helps to "see."
Photo takomabibelot / PD
SAS Institute develops software for statistical data analysis. In most cases, the results of this analysis are displayed as graphs, so it was difficult for visually impaired people to work with SAS solutions.
One of the company's developers , Ed Summers, took over the situation . He himself lost his sight at the age of ten due to a degenerative eye disease (he developed retinitis pigmentosa ).
Ed and colleagues created a solution that would help visually impaired people to use infographics and charts built with SAS applications for educational or business purposes. So appeared Graphics Accelerator . This is a free Google Chrome plugin that uses sonification methods .. It presents graphic information — including heat maps and pie charts — in audio format.
With the function graphs, the tool works as follows: the user moves the pointer along the horizontal axis using the arrows on the keyboard, and the Graphics Accelerator plays a piano note for each point on the graph. The larger the value on the Y axis, the higher the sound. How this sounds can be heard in the video clip of the SAS Institute .
For ease of navigation, values from the left half of the graph are reproduced through the left earpiece speaker (or speaker), and from the right through the right speaker. The graph is also accompanied by textual descriptions that you can listen to using a screen reader.
A person is able to "filter" unnecessary sounds , ignore background noise and focus only on the necessary information. That is why we can easily recognize the voices of people in a lively environment. This property is known as the “ cocktail party effect ”.
Using the methods of sonification, it is easier for us to concentrate and notice certain specific features in the data, to find patterns. For example, NASA developed its own sonification tool xSonify . This program uses the astronomer Wanda Diaz-Merced (Wanda Diaz Merced), in his research.
During the performanceon TED, she said that with the help of a sonification program she analyzed graphs showing the intensity of the star’s light over time, “listening” to gamma-ray bursts of celestial bodies.
Photo by Antonio Roberts / CC
Sonification is widely used by researchers from other areas as well. Researchers from the University of Idaho and Eastern University of Washington studied with it the features of the migration of Chinook salmon - fishes of the salmon family. When the fish moves from the reservoir to the reservoir, it changes the chemical composition of the otoliths , small solid formations in the inner ear. Analyzing this structure, scientists can understand where this or that fish visited.
The researchers collected data on changes in the concentration of each chemical element and compared them with each other. Information presented in the form of sound. Each element in the otolith was assigned its own frequency. When the fish moved from one place to another, the sound that “made” otolith changed.
Methods of sonification are used by people working on projects at the intersection of science and art. The sound helps to visually show the important changes taking place in the world, such as global warming . Dry numbers are transformed into disturbing music, which illustrates the rise in the average temperature on the planet and the level of carbon dioxide in the atmosphere.
Similar projects are being created by developer and artist Brian Foo. With the help of sonification, he showed the financial well-being of people in different parts of New York , the level of air pollution in Beijing , as well as gender and race diversity in Hollywood films .
Sonification allows you to learn more about the "invisible" data that surrounds us every day. For example, with its help you can hear the Internet. In the performance of Internet Rumbles , which was held at the KISS2014 symposium , sounds presented information about the Internet connection of devices in the room. You can also voice changes in the contents of different sites: similar experiments were conducted on the basis of data from Habr and Wikipedia .
Thus, sonification provides an opportunity to go beyond the usual work with information and look at our world from a different, unexpected angle.
More interesting about sound - in our Telegram channel :
How to hear color
Brand history: Marshall
Some facts about audio cassettes
How Star Wars sounded
Where VK music is going
His task is to translate graphic information into sound. Today we talk about the development of SAS Institute and other cases where the sound helps to "see."
Photo takomabibelot / PD
How Graphics Accelerator Works
SAS Institute develops software for statistical data analysis. In most cases, the results of this analysis are displayed as graphs, so it was difficult for visually impaired people to work with SAS solutions.
One of the company's developers , Ed Summers, took over the situation . He himself lost his sight at the age of ten due to a degenerative eye disease (he developed retinitis pigmentosa ).
Ed and colleagues created a solution that would help visually impaired people to use infographics and charts built with SAS applications for educational or business purposes. So appeared Graphics Accelerator . This is a free Google Chrome plugin that uses sonification methods .. It presents graphic information — including heat maps and pie charts — in audio format.
With the function graphs, the tool works as follows: the user moves the pointer along the horizontal axis using the arrows on the keyboard, and the Graphics Accelerator plays a piano note for each point on the graph. The larger the value on the Y axis, the higher the sound. How this sounds can be heard in the video clip of the SAS Institute .
For ease of navigation, values from the left half of the graph are reproduced through the left earpiece speaker (or speaker), and from the right through the right speaker. The graph is also accompanied by textual descriptions that you can listen to using a screen reader.
Where else is sonification used
A person is able to "filter" unnecessary sounds , ignore background noise and focus only on the necessary information. That is why we can easily recognize the voices of people in a lively environment. This property is known as the “ cocktail party effect ”.
Using the methods of sonification, it is easier for us to concentrate and notice certain specific features in the data, to find patterns. For example, NASA developed its own sonification tool xSonify . This program uses the astronomer Wanda Diaz-Merced (Wanda Diaz Merced), in his research.
During the performanceon TED, she said that with the help of a sonification program she analyzed graphs showing the intensity of the star’s light over time, “listening” to gamma-ray bursts of celestial bodies.
Photo by Antonio Roberts / CC
Sonification is widely used by researchers from other areas as well. Researchers from the University of Idaho and Eastern University of Washington studied with it the features of the migration of Chinook salmon - fishes of the salmon family. When the fish moves from the reservoir to the reservoir, it changes the chemical composition of the otoliths , small solid formations in the inner ear. Analyzing this structure, scientists can understand where this or that fish visited.
The researchers collected data on changes in the concentration of each chemical element and compared them with each other. Information presented in the form of sound. Each element in the otolith was assigned its own frequency. When the fish moved from one place to another, the sound that “made” otolith changed.
Methods of sonification are used by people working on projects at the intersection of science and art. The sound helps to visually show the important changes taking place in the world, such as global warming . Dry numbers are transformed into disturbing music, which illustrates the rise in the average temperature on the planet and the level of carbon dioxide in the atmosphere.
Similar projects are being created by developer and artist Brian Foo. With the help of sonification, he showed the financial well-being of people in different parts of New York , the level of air pollution in Beijing , as well as gender and race diversity in Hollywood films .
Sonification allows you to learn more about the "invisible" data that surrounds us every day. For example, with its help you can hear the Internet. In the performance of Internet Rumbles , which was held at the KISS2014 symposium , sounds presented information about the Internet connection of devices in the room. You can also voice changes in the contents of different sites: similar experiments were conducted on the basis of data from Habr and Wikipedia .
Thus, sonification provides an opportunity to go beyond the usual work with information and look at our world from a different, unexpected angle.
More interesting about sound - in our Telegram channel :
How to hear color Brand history: Marshall Some facts about audio cassettes How Star Wars sounded Where VK music is going