Alerting from Nagios over the phone via Asterisk

I read the Google translate + Asterisk IVR article yesterday and thought, “Cool! You can use it to quickly create sound files when starting new hosts and services in Nagios! ”

Then I asked myself, is it described somewhere how to set up telephone alerts for Nagios? Googling a little, I found a couple of articles, read them and realized that the solutions described there have a number of disadvantages:

• Nagios and Asterisk must be installed on the same server.
• Local installation of the engine for speech synthesis is required.
• The administrator’s phone is “wired” into the script and it receives ALL notifications.

So I decided to share my decision, which I have been using for more than a year. In general terms, the solution is as follows. In nagios, contact objects are created with the phone number in the pager parameter and the notify-by-phone and host-notify-by-phone commands that run the /etc/nagios3/notify_by_phone.sh script with parameters for whom to call and which service fell. The script, in turn, passes parameters via ssh to the Asterisk server.

On the Asterisk side, instead of the cache, the script /etc/asterisk/call_from_nagios.sh is run , which generates call filesfor Asterisk with a variable containing a list of phrases to be voiced. That’s basically all, further it is described in detail.

I will not describe the installation process of Nagios and Asterisk here, suppose that both of these services already work for you. Let's start with the part of the Asterisk server. We create the script /etc/asterisk/call_from_nagios.sh :

#!/bin/bash
data=`cat <&0`
_TMP=/var/tmp
callfile="${_TMP}/nag_callfile_$$"
echo "`date '+%Y.%m.%d %H:%M:%S'` ${data}" >> ${_TMP}/nag.log
number=`echo "${data}" | cut -f 1 -d " "`
data=`echo ${data} | cut -f 1 -d ":" | cut -d " " -f 2-100 | tr "[:upper:]" "[:lower:]" | sed 's/ /\&/g' `
echo "Channel: Local/${number}@from-internal
Context: custom-nagios-say
Extension: s
Priority: 1
MaxRetries: 0
WaitTime: 40
Setvar: play=${data}
Account: NAGIOS
CallerId: \"NAGIOS\" <168>
" > $callfile
chmod 666 $callfile
mv $callfile /var/spool/asterisk/outgoing/

Create a context in extensions.conf :

[custom-nagios-say]
exten => s,1,Answer()
exten => s,n,Wait(1)
exten => s,n,Set(CHANNEL(language)=nag)
exten => s,n,Playback(intro)
exten => s,n,Playback(${play})
exten => s,n,Playback(${play})
exten => s,n,Playback(end)
exten => s,n,Hangup()

Create the directory / var / lib / asterisk / sounds / nag / and put the following files in it:
critical.mp3 - critical
down.mp3 - unavailable
end.mp3 - detailed information on email
host.mp3 - server
intro.mp3 - note!
problem.mp3 - problem
service.mp3 - service
status.mp3 - status
warning.mp3 - warning
As well as the names of servers and services. The files themselves can be voiced in your own voice or in the manner described in the topic, which led me to think about this article:

wget -U "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5" "http://translate.google.com/translate_tts?q=сервер&tl=ru" -O host.mp3

There are two further options: install the format_mp3 module in Asterisk or convert each file into a digestible asterisk format. I preferred the first method, but for an example I will give a command for conversion:

sox host.mp3 -r 8000 -c 1 host.wav

Now you can start the test call with the command:

echo "168 PROBLEM Host ISP status DOWN" | /etc/asterisk/call_from_nagios.sh

After you catch all the jambs and get a call and say “Attention! Problem. Host is an internet service provider. Status - unavailable ”, you can go to the next step and prepare the ability to receive alerts over the network via ssh.

We generate a separate ssh certificate without passphrase:

ssh-keygen -t rsa -b 4096

Now we add the contents of id_rsa.pub to /root/.ssh/authorized_keys and add the command parameter to the beginning of the line . Do not forget to do

chmod 600  /root/.ssh/authorized_keys

Our file will look something like this:

command="/etc/asterisk/call_from_nagios.sh" ssh-rsa AAAAB3NпропущеномногобуквA0PCGAC/8kZU= root@nagios

Check again with the next command and move on.

echo "168 PROBLEM Host ISP status DOWN" | ssh -l root -i id_rsa localhost

Now let's work with the Nagios server. Suppose we have some contact:

define contact{
	        contact_name                    vasea
	        alias                           Vasea Pupkin
	        service_notification_period     24x7
	        host_notification_period        24x7
	        service_notification_options    w,u,c,r
	        host_notification_options       d,r
	        service_notification_commands   notify-service-by-email
	        host_notification_commands      notify-host-by-email
	        email                           vasea@mydoamin.ru
	        }

Copy it and modify it a bit:

define contact{
	        contact_name                    vasea_phone
	        alias                           Vasea Pupkin phone
	        service_notification_period     dayhours
	        host_notification_period        dayhours
	        service_notification_options    w,c,r
	        host_notification_options       d,r
	        service_notification_commands   notify-by-phone
	        host_notification_commands      host-notify-by-phone
	        pager                           163
	        }

Here I replaced the notification period with daytime (previously described with the define timeperiod directive ), which corresponds to a time from 7 to 22 hours. If Vasya is ready to receive calls at night, you can leave 24x7. Well, in fact, I added the phone number 168. I use internal phones, and the forwarding to the mobile is already set up in the asterisk itself. But, in principle, you can immediately indicate the mobile number.

Add vasea_phone contact to the list of responsible persons for the corresponding service or to the corresponding contactgroup . Next, create the commands:

define command{
        command_name    notify-by-phone
        command_line    [ "$NOTIFICATIONTYPE$" = "PROBLEM" ] && /etc/nagios3/notify_by_phone.sh "$CONTACTPAGER$ $NOTIFICATIONTYPE$ Host $HOSTNAME$ Service $SERVICEDESC$ status $SERVICESTATE$ : $SERVICEOUTPUT$"
}
define command{
        command_name    host-notify-by-phone
        command_line    [ "$NOTIFICATIONTYPE$" = "PROBLEM" ] && /etc/nagios3/notify_by_phone.sh "$CONTACTPAGER$ $NOTIFICATIONTYPE$ Host $HOSTNAME$ status $HOSTSTATE$ : $HOSTOUTPUT$"
}

And the script /etc/nagios3/notify_by_phone.sh as follows:

#!/bin/bash
data=$@
date=`/bin/date '+%Y.%m.%d %H:%M:%S'`
echo "${date} ${data}" >> /tmp/nag.log
echo "${data}" | ssh -i /etc/nagios3/id_rsa root@10.1.5.61

10.1.5.61 is the IP address of my Asterisk server. And the / etc / nagios3 / id_rsa file is the one we generated on the Asterisk server. When connecting to a new server for the first time, ssh asks for confirmation before entering its fingerprint in known_hosts. Therefore, we need to become a nagios user (if the Nagios daemon runs under this user, this is usually the case) and manually run the /etc/nagios3/notify_by_phone.sh script :

su - nagios
/etc/nagios3/notify_by_phone.sh 168 PROBLEM Host ISP status DOWN

Before doing this, you may need to replace / bin / false with / bin / bash in / etc / password for the nagios user. After executing the above command, this can be returned back.

That's all! Maybe some details I overlooked. If so - write in the comments and I will update the article.

Now you will be the first to know that a problem has occurred and correct it before any of the clients or management has noticed it.