Asterisk speech recognition through Google + smart IVR

Good day, dear Habra users.
In one project, it was necessary to make a smart IVR based on Asterisk IP-PBX. What is meant by the word “smart”: when calling a certain number, the station asks for the name of the subscriber, the person on the other end of the wire calls the name and the station connects it to the desired subscriber.

In my case, I used the ready-made assembly of AsteriskNow with FreePBX already preinstalled, although in this case it does not play a special role, because the differences will be only in editing the dial plan.

Step 1:

Work with google

The first thing is to somehow recognize the caller’s speech. There were enough ( one , two ) articles on the Habr how to do this using Google Translate. I decided to take ready-made scripts found on the expanses of github: googletts.agi - in order to teach Asterisk to speak and speech-recog.agi - so that Asterisk can recognize speech.

Files googletts.agi and speech-recog.agi are thrown into the folder / var / lib / asterisk / agi-bin.
For successful scripts to work, you must have the following packages: Perl, perl-libwww, IO-Socket-SSL, flac, sox, mpg123. All the packages I successfully downloaded and installed from the repositories (via yum install), with the exception of mpg123, I had to download it separately.

In googletts.agi we change the value of the $ lang variable from en to ru, because we want Asterisk to speak Russian.
In speech-recog.agi, we change the value of the $ language variable from en-US to ru-RU so that Google returns the result in Russian.
Everything, I didn’t touch anything else in the scripts.

Step 2:

Writing a dial plan

As I said above, I have FreePBX installed, so I will make all the changes to the extensions_custom.conf file.
To begin with, it's nice to greet the caller and give him a comment on what to do next. Then, using speech-recog.agi, we listen to what the user says, record, convert, send to Google and get the results from him. Next, using the GotoIf function , we check how the script worked. The script returns the following values: status: returns the execution status. 0 means utterance success : string returned by Google confidence: value from 0 to 1, indicating the probability of correct recognition

exten => 100,1,Answer()
exten => 100,n,agi(googletts.agi,”Здравствуйте! После звукового сигнала произнесите имя абонента.”,ru)



exten => 100,n(record),agi(speech-recog.agi,ru-RU)








exten => 100,n,GotoIf($[$["${status}" = "0"] & $["${confidence}" > "0.8"]]?if1:retry)

If the test succeeds, go to the if1 event, if it fails, go to the retry event, in which we ask the user to repeat.

exten => 100,n(retry),agi(googletts.agi,”Пожалуйста, повторите”,ru)

Next, we proceed to work directly with the line itself, which we received from Google. It is necessary to compare the received string $ {utterance} with some template and decide what to do next. We will use the GotoIf function.

exten => 100,n(if1),GotoIf($[“${utterance}” = “вася”]?vasya:retry)

If the line received from Google matches “Vasya”, go to the vasya event, if it does not match, we ask the user to repeat.

And it remains only to call Vasya

exten => 100,n(vasya),Dial(SIP/101)

Variations on a Theme

• For simplicity of the example, I cited only one condition for comparing what they heard with the template, in fact there can and should be more.
• Not very nice phrases that say "iron lady" from Google. In the working version, of course, it’s nice to record phrases with a “live” voice and play them using the Playback function.

Subtleties

With this type of work with Google Translate, it should be borne in mind that it works well, but not completely, and this must be taken into account when creating those templates with which we will compare the result obtained from Google.

Here is an example of a rake that I stepped on:
My name is Cyril (two "L" at the end). Google for some reason only known to him once again returned either “Cyril” or “Cyril”.

Afterword

There is a suspicion that the comparison could be implemented in some more technological way, I will be happy to hear your opinion and suggestions in the comments.
And there remains an open question on a scale: what will happen if there are a lot of subscribers, how long will it take to go through all the comparisons, if, of course, they are implemented using the method I proposed. But for a small PBX for about 20 subscribers, this method is acceptable.

Thanks for attention.

UPD

Examples

As an example, I used a slightly different dial plan, but the essence of this does not change.


Record link with successful recognition.

Link to NOT successful recognition.