Secure SMS Commander
Most system administrators in one way or another monitoring the equipment of the company are familiar with the smstools package, which is often used only to send SMS messages about the state of the infrastructure. The flip side of the coin is the need to remotely control the enterprise infrastructure when the VPN connection is unavailable or if you are outside the Internet access zone. The previously described management methods certainly have the right to life, but are designed to manage a single server and do not provide the ability to safely execute commands. Below in the text, I bring to your attention a working mechanism for securely confirming the execution of sms commands inside the local network.
According to the rule "A properly configured system does not need maintenance" , the correct system administrators do not reconfigure stable operating equipment, limiting themselves only to installing updates.
But in the work of servers there are times when a thread is hung up that ensures the operation of the network ball and RPC, say LanManServer (in the common people server), and not the process itself, but some kind of thread is hung up. As a result, we have a server that cannot be restarted through remote control or correctly repaid.
Almost all modern servers have IPMI and power management on board. From here we will build our functionality. If your servers are additionally monitored through nagios or similar packages, then in most cases you use nrpe - Nagios Remote Plugin Executor , which means you have a server management mechanism independent of system services. It is enough to specify frequently used commands like reboot, shutdown, stop_server, start_server in the NSClient ++ config and remotely do the necessary actions with the server.
In the old client config, the call to external scripts is located in the " [External Scripts] " section , in the new " [/ settings / external scripts / scripts] ". The text is the same in both cases.
tekstovka scripts
s_reboot.cmd
s_shutdown.cmd
start_server.cmd
stop_server.cmd
This is a basic list of commands that can be used remotely. You can cause a service to stop or raise by passing arguments to a similar script. For example, distort IIS.
Calling a team is usually simple and looks like
It seems to be sorted out with nrpe, now you need to serve IPMI.
For dialogue with the IPMI server system, you can use the ipmitool package , which is an interface to access ipmi management from the command line.
The main set of commands that we need relates to the “power” section, namely
You can still read the temperature sensors, the fan speed, but we don’t really need it.
With the main set of teams sorted out.
Due to the fact that recently a number of services have appeared that allow forging SMS sender number - the usual verification of the sender number has become not reliable enough.
Forwarding a direct password in a message is also an unreliable thing, because interception of the message will allow to get full access to your control system. What to do?
The solution is surprisingly simple. OTP (One Time Passwords). From the point of view of ease of use and the absence of dependence of the generated passwords on time, the optimal solution would be to use OPIE .
If you have several system administrators or engineers serving the server, then giving everyone a password and the initial initialization vector will not be entirely correct. For server maintenance, it is appropriate to give everyone a list of passwords from different blocks.
And in the case of using password blocks, you can accurately confirm with whose hand the server was turned off or restarted. From the point of view of using the “one-person” system, for Android phones, it would be more appropriate to install the OTPDroid package for online password generation. It is wonderful in that the value of the generated password is immediately copied to the clipboard and three clicks are enough to copy the password in sms.
You ask, “What about the double use of the same password?”
The answer is simple: "You just need to control and maintain a list of used sequences that arrived in SMS"
The first step in the smsd configuration file is to add an event handler call.
The processing program is a regular shell script.
A few comments about the smsevent handler file.
At the beginning of the file, basic variables are set for finding the configuration file, verification and management programs, and OPIE parameters for password verification.
In the configuration file, you specify the phones and a list of commands available to them, a list of server addresses, IPMI logins and passwords for various hosts, and templates for running commands.
In the command template block, the second argument is 0 or 1 and either skips the OPIE check for the command or checks the OPIE.
The simplest SMS request will be sent back alternately 2 sms with the results of checking host availability:
A request of the form will send the host 192.168.55.2 to reboot by emulating the RESET button, and the host 192.168.54.5 will shut down normally:
All actions performed via sms are written to the extended smsd log.
In general, that's all. You edit the config for yourself, configure the server to receive commands and you can go on vacation with the phone. If suddenly something hangs, direct-fire shot via sms will do the trick.
PS: The script is a little damp, but as they say "there is no limit to perfection!".
To begin with, “Why is this needed?”
According to the rule "A properly configured system does not need maintenance" , the correct system administrators do not reconfigure stable operating equipment, limiting themselves only to installing updates.
But in the work of servers there are times when a thread is hung up that ensures the operation of the network ball and RPC, say LanManServer (in the common people server), and not the process itself, but some kind of thread is hung up. As a result, we have a server that cannot be restarted through remote control or correctly repaid.
What can be done with this problem?
Almost all modern servers have IPMI and power management on board. From here we will build our functionality. If your servers are additionally monitored through nagios or similar packages, then in most cases you use nrpe - Nagios Remote Plugin Executor , which means you have a server management mechanism independent of system services. It is enough to specify frequently used commands like reboot, shutdown, stop_server, start_server in the NSClient ++ config and remotely do the necessary actions with the server.
In the old client config, the call to external scripts is located in the " [External Scripts] " section , in the new " [/ settings / external scripts / scripts] ". The text is the same in both cases.
reboot=scripts\s_reboot.cmd
shutdown=scripts\s_shutdown.cmd
stop_server=scripts\stop_server.cmd
start_server=scripts\start_server.cmd
tekstovka scripts
s_reboot.cmd
@echo "Reboot initiated"
@start cmd /c shutdown -r -f -t 00
@exit 0
s_shutdown.cmd
@echo "Shutdown initiated"
@start cmd /c shutdown -s -f -t 00
@exit 0
start_server.cmd
@start cmd /c net start server
@if %ERRORLEVEL% EQU 0 goto ok
@echo Some problem with service start
@exit 1
:ok
@echo Service started
@exit 0
stop_server.cmd
@start cmd /c net stop server /y
@if %ERRORLEVEL% EQU 0 goto ok
@echo Some problem with service stop
@exit 1
:ok
@echo Service stopped
@exit 0
This is a basic list of commands that can be used remotely. You can cause a service to stop or raise by passing arguments to a similar script. For example, distort IIS.
Calling a team is usually simple and looks like
/usr/local/libexec/nagios/check_nrpe2 -H -c -a
It seems to be sorted out with nrpe, now you need to serve IPMI.
For dialogue with the IPMI server system, you can use the ipmitool package , which is an interface to access ipmi management from the command line.
The main set of commands that we need relates to the “power” section, namely
ipmitool -H -U -P power status - узнать состояние питания на сервере
ipmitool -H -U -P power up - включить сервер
ipmitool -H -U -P power off - выключить питание
ipmitool -H -U -P power reset - эмулировать нажатие клавиши reset
You can still read the temperature sensors, the fan speed, but we don’t really need it.
With the main set of teams sorted out.
How to ensure management security?
Due to the fact that recently a number of services have appeared that allow forging SMS sender number - the usual verification of the sender number has become not reliable enough.
Forwarding a direct password in a message is also an unreliable thing, because interception of the message will allow to get full access to your control system. What to do?
The solution is surprisingly simple. OTP (One Time Passwords). From the point of view of ease of use and the absence of dependence of the generated passwords on time, the optimal solution would be to use OPIE .
Why OPIE?
If you have several system administrators or engineers serving the server, then giving everyone a password and the initial initialization vector will not be entirely correct. For server maintenance, it is appropriate to give everyone a list of passwords from different blocks.
# opiekey -5 -n 5 5 habrahabr
Using the MD5 algorithm to compute response.
Reminder: Dont use opiekey from telnet or dial-in sessions.
Enter secret pass phrase:
1: JIG RIFT BODE OLGA RICK JAG
2: RIM HIVE BANG LIMA HELL OMEN
3: BULB MOD CARR BANK MOS SET
4: GARB BAWL MANY HAL GLOW FEED
5: FAWN EDGY MEET SHUT LIKE TIME
# opiekey -5 -n 5 25 habrahabr
Using the MD5 algorithm to compute response.
Reminder: Dont use opiekey from telnet or dial-in sessions.
Enter secret pass phrase:
21: MAIN HOFF JAM OATH SMOG LIED
22: FUND DENY BYTE BOLT NIBS EASY
23: SLY COAT FLEA CAGE MAE COAL
24: SURE LEFT HULK CLAN SHUN DAR
25: GRAB LIE CLAN FLAK MEL ROSE
And in the case of using password blocks, you can accurately confirm with whose hand the server was turned off or restarted. From the point of view of using the “one-person” system, for Android phones, it would be more appropriate to install the OTPDroid package for online password generation. It is wonderful in that the value of the generated password is immediately copied to the clipboard and three clicks are enough to copy the password in sms.
You ask, “What about the double use of the same password?”
The answer is simple: "You just need to control and maintain a list of used sequences that arrived in SMS"
How to make all this work together?
The first step in the smsd configuration file is to add an event handler call.
eventhandler = /usr/local/etc/smshandlers/smsevent
The processing program is a regular shell script.
Next will go a lot of smsevent shell code
#!/bin/sh
NRPETOOL="/usr/local/libexec/nagios/check_nrpe2"
IPMITOOL="/usr/local/bin/ipmitool"
FPINGTOOL="/usr/local/sbin/fping"
PARAMFILE="/usr/local/etc/smshandlers/param.list"
OPIEKEYBIN="/usr/bin/opiekey"
OPIESEED="HABROPIEProc"
OPIEPASS="StrongOPIEPassword"
OPIEUSEDSEED="/var/tmp/used.opie"
SMSDOUTSPOOL="/var/spool/sms/outgoing"
if [ ! -f ${OPIEUSEDSEED} ]; then
echo "#USED SEEDS FILE. Do not EDIT" > ${OPIEUSEDSEED}
fi
if [ ! -w ${OPIEUSEDSEED} ]; then
echo "file ${OPIEUSEDSEED} is not writable"
exit
fi
if [ ! -r ${PARAMFILE} ]; then
echo "file ${PARAMFILE} is not readable"
exit
fi
IFS="
"
FOUNDNUM=""
NUMBER=""
# search any properties
search (){
# $1 - section [name] from param.list
# $2 - search string in section
SECTION=$1
SEARCHSTR=$2
shift
shift
OLDIFS=${IFS}
IFS="
"
[ "X"${SEARCHSTR} == "X" ] && exit
for i in ${PARAMS}; do
# search section definition
if [ `echo ${i} | grep -e "^\["` ]; then
CURSECTION=${i}
continue
fi
if [ "${CURSECTION}" == "["${SECTION}"]" ]; then
if [ `echo ${i} | grep -e "^\b${SEARCHSTR}\b:"` ]; then
echo ${i}
break
fi
fi
done
IFS=${OLDIFS}
}
cut_message_body (){
# split sms file to parts
# BSD buggy "grep -m 1 -A 10 -e '^$'" replacement
#
FILE=$1
NULLSTR=0
for i in `cat -e $1`; do
if [ "X"${i} == "X\$" ]; then
NULLSTR=1
continue
fi
if [ ${NULLSTR} -eq 1 ]; then
echo ${i}
fi
done
}
process_command () {
CMD=$1
shift
CMDPARAM=$*
oIFS=$IFS
IFS=" "
for host in ${CMDPARAM}; do
HOSTACCOUNT=$( search hosts "${host}" )
if [ "X"${HOSTACCOUNT} != "X" ]; then
ACCOUNT=`echo ${HOSTACCOUNT} | cut -d ":" -f 2-`
LOGINPW=$( search logins "${ACCOUNT}" )
if [ "X"${ACCOUNT} != "X" ]; then
if [ "${ACCOUNT}" != "null" ]; then
USER=`echo ${LOGINPW} | cut -d ":" -f 2`
PASSWORD=`echo ${LOGINPW} | cut -d ":" -f 3`
else
USER="USER"
PASSWORD="PASSWORD"
fi
else
echo "ACCOUNT ${ACCOUNT} NOT FOUND!"
break
fi
else
echo "HOST ${host} NOT FOUND!"
continue
fi
opierequired=$( search commands_template "${CMD}" | cut -d ":" -f 2 )
commandtemplate=$( search commands_template "${CMD}" | \
cut -d ":" -f 3- | sed \
-e "s@\%HOST\%@${host}@g" \
-e "s@\%FPING\%@${FPINGTOOL}@g" \
-e "s@%IPMITOOL%@${IPMITOOL}@g" \
-e "s@%CHECK_NRPE%@${NRPETOOL}@g" \
-e "s@%USER%@${USER}@g" \
-e "s@%PASSWORD%@${PASSWORD}@g" )
if [ "${opierequired}" -eq "1" ]; then
if [ "${OPIERES}" == "True" ]; then
echo "OPIE Success"
res=$( eval ${commandtemplate} )
sendsms ${NUMBER} ${host} ${res}
else
echo "OPIE Failed"
sendsms ${NUMBER} ${host} "OPIE Challenge failed"
fi
else
res=$( eval ${commandtemplate} )
sendsms ${NUMBER} ${host} ${res}
fi
done
IFS=$oIFS
}
sendsms () {
NM=$1
shift
TMPFILE=`mktemp ${SMSDOUTSPOOL}/smscmd.XXXXXX` || exit 1
echo "To: ${NM}" >> ${TMPFILE}
echo >> ${TMPFILE}
echo "$*" >> ${TMPFILE}
}
opiecheck () {
OPIESEQ=`echo $* | cut -d " " -f 1 | sed -e 's/[^0-9]//g'`
OPIESTR=`echo $* | cut -d " " -f 2-`
if [ "X"${OPIESEQ} == "X" ]; then
echo "OPIE SEQUENCE FOR NUMBER ${NUMBER} IS NOT SET!"
OPIERES="False"
break
fi
if [ `grep "^${OPIESEQ}$" ${OPIEUSEDSEED}` ]; then
echo "USED OPIE SEQUENCE ${OPIESEQ} DETECTED"
OPIERES="False"
sendsms ${NUMBER} "OPIE SEQUENCE ${OPIESEQ} ALREADY USED"
exit 1
else
echo ${OPIESEQ} >> ${OPIEUSEDSEED}
fi
OPIEGEN=`echo ${OPIEPASS} | ${OPIEKEYBIN} -5 -a ${OPIESEQ} ${OPIESEED} 2>/dev/null`
if [ "X"${OPIESTR} != "X"${OPIEGEN} ]; then
OPIERES="False"
else
OPIERES="True"
fi
}
parse_commands () {
for command in $*; do
command=`echo ${command} | sed -e 's/\$$//g' -e 's/^\#//g'`
CURCOMMAND=`echo ${command} | cut -d " " -f 1`
CMDPARAM=`echo ${command} | cut -d " " -f 2-`
case "${CURCOMMAND}" in
"OPIE")
opiecheck ${CMDPARAM}
continue
;;
*)
if [ `echo ${FOUNDNUM} | cut -d : -f 2 | grep "\b${CURCOMMAND}\b"` ]; then
TMPMSG="ALLOWED COMMAND FOR NUMBER "${NUMBER}" - COMMAND: "${CURCOMMAND}" ARGS "${CMDPARAM}
echo ${TMPMSG}
process_command ${CURCOMMAND} ${CMDPARAM}
else
TMPMSG="DISALLOWED COMMAND FOR NUMBER "${NUMBER}" - COMMAND: "${CURCOMMAND}" ARGS "${CMDPARAM}
echo ${TMPMSG}
fi
;;
esac
done
}
PARAMS=`cat ${PARAMFILE} | grep -E -v '^#'`
if [ $1 == "RECEIVED" ]; then
FILENAME=$2
NUMBER=`grep 'From:' ${FILENAME} | sed -e 's/^From\:\ //g' -e 's/[^0-9]//g;'`
FOUNDNUM=$( search phones ${NUMBER} )
if [ "X"${FOUNDNUM} != "X" ]; then
commands=$( cut_message_body ${FILENAME} )
parse_commands ${commands}
fi
elif [ $1 == "SENT" ]; then
echo $@
else
echo "UNKNOWN STATUS $@"
fi
A few comments about the smsevent handler file.
At the beginning of the file, basic variables are set for finding the configuration file, verification and management programs, and OPIE parameters for password verification.
A bit of param.list configuration file
# ---------------------------------------
#
# ALLOWED PHONES
# PHONENUMBER:COMMAND,COMMAND,COMMAND
# ---------------------------------------
[phones]
79030000000:RESET,PWROFF,PWRON,REBOOT,SHUTDOWN,STOPSERVER,STARTSERVER,ALIVE,PWRSTAT
79160000000:ALIVE,PWRSTAT
79020000000:PWRON,STARTSERVER,ALIVE
# ---------------------------------------
#
# HOST LIST
# REAL_IP:IPMI_USER
# ---------------------------------------
[hosts]
192.168.55.2:ADMSM
192.168.54.2:null
192.168.55.5:ADMDEF
192.168.54.5:null
# ---------------------------------------
#
# IPMI LOGINS
# IPMI_ACCOUNT:IPMI_NAME:IPMI_PASSWORD
# ---------------------------------------
[logins]
ADMSM:ADMIN:PW1234
ADMDEF:ADMIN:ADMIN
ADMIBM:USERID:USERID
# ---------------------------------------
#
# COMMANDS TEMPLATE
# COMMAND:OPIE_CHECK:COMMAND_TEXT
# ---------------------------------------
[commands_template]
RESET:1:%IPMITOOL% -H %HOST% -U %USER% -P %PASSWORD% power reset
PWROFF:1:%IPMITOOL% -H %HOST% -U %USER% -P %PASSWORD% power off
PWRON:0:%IPMITOOL% -H %HOST% -U %USER% -P %PASSWORD% power on
PWRSTAT:0:%IPMITOOL% -H %HOST% -U %USER% -P %PASSWORD% power status
REBOOT:1:%CHECK_NRPE% -H %HOST% -p 5666 -c reboot
SHUTDOWN:1:%CHECK_NRPE% -H %HOST% -p 5666 -c shutdown
STOPSERVER:1:%CHECK_NRPE% -H %HOST% -p 5666 -c stop_server
STARTSERVER:0:%CHECK_NRPE% -H %HOST% -p 5666 -c start_server
ALIVE:0:%FPING% %HOST%
In the configuration file, you specify the phones and a list of commands available to them, a list of server addresses, IPMI logins and passwords for various hosts, and templates for running commands.
In the command template block, the second argument is 0 or 1 and either skips the OPIE check for the command or checks the OPIE.
The simplest SMS request will be sent back alternately 2 sms with the results of checking host availability:
ALIVE 192.168.54.2 192.168.54.5
A request of the form will send the host 192.168.55.2 to reboot by emulating the RESET button, and the host 192.168.54.5 will shut down normally:
OPIE 1 COT NORM TILL JILL MOST MESH
RESET 192.168.55.2
SHUTDOWN 192.168.54.5
All actions performed via sms are written to the extended smsd log.
In general, that's all. You edit the config for yourself, configure the server to receive commands and you can go on vacation with the phone. If suddenly something hangs, direct-fire shot via sms will do the trick.
PS: The script is a little damp, but as they say "there is no limit to perfection!".