Capture and visualize! Or a histogram from a microphone using the Web Audio API

I really like live graphics. Deadly boredom - look at static pictures with numbers. I want the chart to be fascinating, to make the person who is looking at it interact and discover new facets of all the data on it. Therefore, any example that falls into my hands, and any visualization library that was not lucky to be on my machine, passes the test of "animation". So once again, thinking about how else I can razoryachit visualization widgets from the DevExtreme library, I thought about displaying sound. “Interesting and lively” - I thought that day, stocked up tea with cookies and sat down for this task. What I ended up with - you'll find out under the cut.

The first thing I had to choose was what sound to visualize. The melody just seemed too banal, besides, I found a wonderful article on Habré that fully describes the whole process from creating sound to rendering on Canvas. But after reading the comments on this article, I noticed a certain interest in capturing audio from a microphone. Having poked Google about the availability of such articles, I found out that there are few of them on this topic. A goal appeared - an action plan appeared:



Capture audio from a microphone . One of the easiest ways to get microphone data in a browser is to use the getUserMedia method, which is in the object of public browser methods - in navigator. He accepts three arguments:

1. Audio and video capture settings:
   

   { audio: true/false, video: true/false }
   

   
   
2. Success Function
3. Function performed in case of error

That is, in general, a call to this method looks like this:

navigator.getUserMedia(
    { audio: true, video: true },
    function (stream) {
        //stream processing
    },
    function (error) {
        //error processing
    }
);

But! It must be remembered that the method is different in different browsers, so it will not be superfluous to turn the following thing before calling getUserMedia, and provide support for the method in all common browsers:

navigator.getUserMedia = (navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia);

Analysis of captured audio . If the getUserMedia method is successful, we get an object of type MediaStream, which can already be used to create an audio source using the Web Audio API. But to use this method, first you need to create an AudioContext - our guide to the world of Web Audio API. The creation is very simple:

var ctx = new AudioContext();

Again, do not forget about cross-browser compatibility:

var AudioContext = window.AudioContext || window.webkitAudioContext;

Now, with the help of the ctx audio context, it is possible to create all the necessary elements for analyzing our sound. What are these elements?



The Destination element is available to us by default, after creating the audio context. In fact, this is where our sound will go, and using the ctx.destination construct gives us access to the standard sound output for the system, for example, speakers or headphones. The remaining elements from this scheme must be created and configured.

• Source is the sound source, our caught audio from the microphone. You just need to convert the MediaStream stream to an element with which we can work, and the createMediaStreamSource audio context method is suitable for this:
  

  var source = ctx.createMediaStreamSource(stream);
  

  
  
• Analyser is a node for audio analysis; many sound parameters can be pulled out with its help. It is created using the simple audio createAnalyser context method:
  

  var analyser = ctx.createAnalyser();
  

  
  
• ScriptProcessor is an audio processing module. It is necessary to track the moment of sound change using the onaudioprocess event. You can create it using the createScriptProcessor method of the audio context with the parameters of the buffer size, the number of inputs and outputs:
  

  var processor = ctx.createScriptProcessor(2048, 1, 1);
  

  
  

All created elements must be interconnected, as shown in the diagram. The Web Audio API provides a couple of connect / disconnect methods for linking and untying nodes:

source.connect(analyser);
source.connect(processor);
analyser.connect(ctx.destination);
processor.connect(ctx.destination);

When you run all this code ( here is an example ) and with a little bit of luck, you can hear sound from the speakers from the speakers / headphones. Listening to your voice is very annoying, so with a clear conscience you can not create a connection between the analyzer and the audio output, it will not affect the final result. The final scheme of work will look like this:



The final touch - parsing sound by bricks. It is possible for the analyzer to set the dimension of the Fourier transform — the fftSize option. Why did she give up? And then, as a result, we get the number of data elements equal to fftSize / 2, that is, the number of points for the graph:

analyser.fftSize = 128;

It remains only to receive data on sound frequencies from the analyzer, storing them in the data variable specially created for this. Sound changes are monitored thanks to the onaudioprocess event we already know:

var data = new Uint8Array(analyser.frequencyBinCount);
processor.onaudioprocess = function (){
    analyser.getByteFrequencyData(data);
    console.log(data);
}

Ta Dam! And here they are, the cherished digits from the microphone in the console ( example ):



Visualization of the received data . Directly “revitalizing” part, turning the numbers into a graph, for which you need the DevExtreme library . The first thing, of course, is to create a schedule:

var chart = chart = $("#chart").dxChart({ 
    //chart options
}).dxChart("instance");

Such a created chart will be empty, because it needs data - the dataSource option, and the series that needs to be displayed - the series option. At the same time, the data is dynamic, and they need to be changed when the onaudioprocess event is triggered:

var dataSource = $.map(data, function (item, index) {
    return { arg: index, val: item };
});
chart.option("dataSource", dataSource);

Adding a couple of settings for the graph, such as color and type of series, you can end up with such a wonderful thing:

And the final code will look like this, simple and concise:

Here you can find a ready-made example . If you wish, you can collect an example locally, or even replace the visualization with any other one you like. The main thing is how quickly and simply using the Web Audio API and the DevExtreme library a similar example is collected.

As always, I look forward to your comments and suggestions in the comments to the article. Have a nice day, everyone!