September 18, 2015 at 16:19

Monitoring Blacklist IPs with Zabbix

Bit of theory


Public spam databases or “black lists” of IP addresses contain information about IP, which for some reason has been recognized as unfriendly to users. We will not delve into the technological subtleties; it is important that mail programs and services use information from these and their own databases in order to protect recipients' electronic mailboxes from unwanted mailing, from spam.

The essence of the problem


If your IP address is blacklisted, recipients will not receive your emails.

IP entry into public spam databases threatens corporate mail collapse. This is unpleasant even if the e-mail addresses on the domain are only 5 and all users can be temporarily “transplanted” to “regular” mail on Yandex or Mail.ru. But, when more than 50 mailboxes integrated with the CRM system are “pinned” to the internal mail server, the problem becomes catastrophic.

Decision


“Know to anticipate; foresee to manage. " A. Comte The

impending threat can and should be calculated before the client feels the punitive measures of the mail servers and the sales department panicked the local administrator. To do this, we created a script for automatic monitoring of IP, notifying the user about possible problems. Powershell was chosen as the monitoring platform, and zabbix 2.4 was used for notification.

A little bit about creating a script


The most obvious option seemed to be parsing web pages of famous IP search engines in the mxtoolbox and 2ip.ru databases. To solve this problem, .NET and XPATH technologies were used - the most effective tool for parsing XML and WEB, especially for regular expression haters. The first pancake, in a way, turned out to be lumpy, and after a long and exhausting testing, we decided to abandon this decision. The reason was, first of all, the dependence on a third-party service: if the creators of the site change the structure of the code, then in the best case it will be necessary to re-test the software, in the worst - to rewrite the code.

After a thorough analysis of the technologies and developments existing in this field, we came to a more interesting and practical solution. Software development based on powershell 3.0, which will independently check for IP in public spam databases.

How does it work?


Checking the IP address for the presence in the blacklist (DNSBL) is carried out as follows: the checked IP is indicated in the DNS PTR notation (that is, vice versa, “front to back”) and the domain name of the DNSBL server is added. If a response from the server is received, then the verified address is blocked: that is, the IP is seen in one or more black lists. Regardless of the specifics of the response (it can be any), its fact itself indicates that IP is in a spam database.

Some code
Script code:
<#
Функция zabbix для отправки данных. Данная функция принимает на вход параметры:
Key - имя ключа, который создается в элементе данных zabbix
Также необходимо задать переменные
$Zabbix_send = указать где находится утилита zabbix_sender
    $zabbix_port = порт zabbix сервера
    $zabbix_server = адрес zabbix сервера
    $zabbix_host = Имя созданого узла сети в zabbix (чувствителен к регистру)
#>
Function Send_Zabbix {
    param (
        [parameter(mandatory=$true)]
        $key,
        [parameter(mandatory=$true)]
        [int64]$value
    )
    $Zabbix_send = "C:\zabbix_sender.exe"
    $zabbix_port = "10051"
    $zabbix_server = "10.0.33.31"
    $zabbix_host = "Blacklistcheck"
    echo "$key : $value"
                $cmd = "$Zabbix_send -z $zabbix_server -p $zabbix_port -s $zabbix_host  -k `"$key`" -o $value"
                echo "$cmd"
    Invoke-Expression $cmd 
} 
# Место хранения списка адресов для проверки, в формате name,ip
$FilePath = "D:\powershell\blacklist.csv"
# импортируем список в текстовый массив
$iparr = Import-Csv $FilePath
foreach ($ipaddr in $iparr)
{
[string]$IP = $ipaddr.adress
[string]$organization = $ipaddr.organization
$reversedIP = ($IP -split '\.')[3..0] -join '.'
 # заготовленый список спамбаз, по которым будет проходить проверка
$blacklistServers = @(
    "b.barracudacentral.org";
"bl.deadbeef.com";
"bl.emailbasura.org";
"bl.spamcannibal.org";
"bl.spamcop.net";
"blackholes.five-ten-sg.com";
"blacklist.woody.ch";
"bogons.cymru.com";
"cbl.abuseat.org";
"cdl.anti-spam.org.cn";
"combined.abuse.ch";
"combined.rbl.msrbl.net";
"db.wpbl.info";
"dnsbl-1.uceprotect.net";
"dnsbl-2.uceprotect.net";
"dnsbl-3.uceprotect.net";
"dnsbl.cyberlogic.net";
"dnsbl.inps.de";
"dnsbl.njabl.org";
"dnsbl.sorbs.net";
"drone.abuse.ch";
"drone.abuse.ch";
"duinv.aupads.org";
"dul.dnsbl.sorbs.net";
"dul.ru";
"dyna.spamrats.com";
"dynip.rothen.com";
"http.dnsbl.sorbs.net";
"images.rbl.msrbl.net";
"ips.backscatterer.org";
"ix.dnsbl.manitu.net";
"korea.services.net";
"misc.dnsbl.sorbs.net";
"noptr.spamrats.com";
"ohps.dnsbl.net.au";
"omrs.dnsbl.net.au";
"orvedb.aupads.org";
"osps.dnsbl.net.au";
"osrs.dnsbl.net.au";
"owfs.dnsbl.net.au";
"owps.dnsbl.net.au";
"pbl.spamhaus.org";
"phishing.rbl.msrbl.net";
"probes.dnsbl.net.au";
"proxy.bl.gweep.ca";
"proxy.block.transip.nl";
"psbl.surriel.com";
"rbl.interserver.net";
"rdts.dnsbl.net.au";
"relays.bl.gweep.ca";
"relays.bl.kundenserver.de";
"relays.nether.net";
"residential.block.transip.nl";
"ricn.dnsbl.net.au";
"rmst.dnsbl.net.au";
"sbl.spamhaus.org";
"short.rbl.jp";
"smtp.dnsbl.sorbs.net";
"socks.dnsbl.sorbs.net";
"spam.abuse.ch";
"spam.dnsbl.sorbs.net";
"spam.rbl.msrbl.net";
"spam.spamrats.com";
"spamlist.or.kr";
"spamrbl.imp.ch";
"t3direct.dnsbl.net.au";
"tor.dnsbl.sectoor.de";
"torserver.tor.dnsbl.sectoor.de";
"ubl.lashback.com";
"ubl.unsubscore.com";
"virbl.bit.nl";
"virus.rbl.jp";
"virus.rbl.msrbl.net";
"web.dnsbl.sorbs.net";
"wormrbl.imp.ch";
"xbl.spamhaus.org";
"zen.spamhaus.org";
"zombie.dnsbl.sorbs.net"
)
$blacklistedOn = @()
foreach ($server in $blacklistServers)
{
    $fqdn = "$reversedIP.$server"
    try
    {
        $null = [System.Net.Dns]::GetHostEntry($fqdn)
        $blacklistedOn += $server
    }
    catch { }
}
if ($blacklistedOn.Count -gt 0)
{
    Write-Host "$organization is blacklisted on the following servers: $($blacklistedOn -join ', ')"
    send_zabbix -key $organization -value 1
}
else
{
    Write-Host "$organization is not currently blacklisted on any server."
    send_zabbix -key $organization -value 0
    }
  }


In zabbix, data is transmitted through a trapper. To use the trapper data element, we must:

- have a trapper data element configured in Zabbix
- send data to Zabbix.

Create the blacklistcheck host:



Now you need to configure the trapper:

Create the data item.

Name - can be any
Key - the main element when creating a trapper, this element is case sensitive. Key KEY and key are two different keys.
Type of information : there are 3 options suitable for our task: number, symbol or text. But, since we excluded regular expressions, we preferred an integer value to text arrays.
Data type- logical; 2 IP values ​​are important to us: present (1), or not present in spam lists (0).
The display of values can be left as is, but you can create your own transformation, approximately of the following content: 1 = ip is listing, 0 = ip isn`t listing - these values ​​do not affect the script, but the visualization of information in zabbix is ​​interesting.



After that, the data item is ready to accept values. You can check through the zabbix_sender program.

In case of correct operation, a similar result will be obtained:

Zabbix sender result
C: \> zabbix_sender.exe -z 10.0.33.31 -p 10051 -s Blacklistcheck -k fn -o 1 -vv
zabbix_sender.exe [14276]: DEBUG: answer [{"response": "success", "info": “Processe
d: 1; failed: 0; total: 1; seconds spent: 0.000066 "}]
info from server:" processed: 1; failed: 0; total: 1; seconds spent: 0.000066 "
sent: 1; skipped: 0; total: 1

The –vv option is used to view diagnostic data.
The speed of checking one IP is no more than 3 minutes.
Next, you need to configure the trigger for the notification.



You can specify any name. The expression corresponds to a change in the state of a data item from 0 to 1. In case of a trigger, a notification will appear in the main zabbix panel.



Practice use


We have created a compact, well-functioning script that automatically notifies the client of possible problems in the operation of the mail system. Agree that the prevention of this issue is much nicer and calmer compared to the need to solve the problem of "fallen off" mail in time trouble.

We have implemented this solution with our customers, and it shows itself well - especially in companies where the sales department works through commercial mailings. This script allows you to quickly localize the problem that caused IP to get into the spam database, and continue to work without damage.

We wish the number of take-offs of emails coincide with the number of touchdowns!
Tags:

Also popular now: