October 2, 2008 at 10:48Cheaply and angrily block 85% of spam: greysmtpd
Until I became a freelancer, I did not know what spam was. I was smart enough not to shine my email anywhere, but to use temporary email to register on sites. And I'm used to the fact that all my mail runs very fast, letters are never lost, and no one complains that my server refused to accept his letter. Having become a freelancer, I was forced to publish my email, and on many sites - because new customers should be able to easily contact me. And spam poured ... at first a small stream, but lately it has been reminiscent of a full-flowing river rather than a brook.
Using traditional spam filters (RBL, GMail, SpamAssassin, etc.) didn’t work for me: all of them are mistaken from time to time, and they accept normal letters as spam. It doesn’t matter if they crush them, return them to the sender, or store them in a separate folder so that I periodically look at them - in any case, there is a non-zero chance of losing an important letter.
For several years, the problem was solved using the utility deliverI wrote for the convenience of filtering mail. Perl programmers using qmail should like it: in fact, filtering is performed using Perl regular expressions, conveniently applied to the desired parts of the message (a filter is a Perl script :)). I analyzed the received spam with my eyes, and carefully wrote regular expressions for it, which 100% would not coincide with normal letters.
But lately, spam has been adapting to my interests, and masquerading as an IT theme. And well, the dog disguises itself - so good that I can no longer confidently write regexp, which even by chance does not coincide with normal writing. (Note, I clearlyto distinguish specific spam by manual regexp, actually even a specialized Perl script, is no longer able, how can SpamAssassin associates do this?)
I had to look for an alternative approach to this problem ... and it was found, it's greylisting ! Unfortunately, finding the greylisting implementation for qmail was not easy. In principle, there are several of them, but they either require a qmail patch, either are written on the knee, or are heavily overloaded with additional functionality. And I really do not like it, I prefer simple, reliable and safe software - for example, such as DJ Bernstein writes .
As a result, I first set qgreylistrbl- test greylisting effectiveness. And then, admiring the result (but not the implementation) of qgreylistrbl, wrote greysmtpd .
Installing greysmtpd is very simple - it’s one of those that I “stuck and forgot”, you don’t even need to configure anything. The big plus of this approach is that spam instantly becomes about 85% less, and at the same time you are guaranteed not to lose a normal message. And if (when) spammers adapt to greylisting, then its effectiveness will drop, but there will still be no harm from it. In general, this is a must-have thing!
It should be noted that greysmtpd is optimized in order to minimize delays in delivering normal mail - in the vast majority of cases, normal mail will be delivered without any delay at all, as if greysmtpd were not used at all. This is achieved by using RBL and the built-in dynamic host identifier (DSL, dial-up, etc.): if the sender IP of the mail is not blocked in RBL and the sender domain (PTR) does not look like a dynamic host, mail from it is skipped immediately, without using greylisting. (Yes, I use RBL, but not to block mail, but just to force greylisting. For all of us, from time to time, we get to RBL.) In general, we get a kind of “soft greylisting” - a minimum of interference with normal operation at an excellent percentage blocked spam.
I tried to make the script itself reliable, simple and efficient. For example, if any error occurs in the script, it simply skips the current message, as if greysmtpd were not installed at all. SMTP Implementation lapped one-to-one with djb'shnoy utility rblsmtpd . And for efficiency, DNS queries are performed asynchronously. Well, plus all the possible optimizations that make it possible to avoid queries in RBL altogether, because RBL thing is braked. The script size is 200 lines, 5.5KB.
License, as usual, public domain.
Download: greysmtpd , deliver .
Criticism and code review are welcome. :)
Using traditional spam filters (RBL, GMail, SpamAssassin, etc.) didn’t work for me: all of them are mistaken from time to time, and they accept normal letters as spam. It doesn’t matter if they crush them, return them to the sender, or store them in a separate folder so that I periodically look at them - in any case, there is a non-zero chance of losing an important letter.
For several years, the problem was solved using the utility deliverI wrote for the convenience of filtering mail. Perl programmers using qmail should like it: in fact, filtering is performed using Perl regular expressions, conveniently applied to the desired parts of the message (a filter is a Perl script :)). I analyzed the received spam with my eyes, and carefully wrote regular expressions for it, which 100% would not coincide with normal letters.
But lately, spam has been adapting to my interests, and masquerading as an IT theme. And well, the dog disguises itself - so good that I can no longer confidently write regexp, which even by chance does not coincide with normal writing. (Note, I clearlyto distinguish specific spam by manual regexp, actually even a specialized Perl script, is no longer able, how can SpamAssassin associates do this?)
I had to look for an alternative approach to this problem ... and it was found, it's greylisting ! Unfortunately, finding the greylisting implementation for qmail was not easy. In principle, there are several of them, but they either require a qmail patch, either are written on the knee, or are heavily overloaded with additional functionality. And I really do not like it, I prefer simple, reliable and safe software - for example, such as DJ Bernstein writes .
As a result, I first set qgreylistrbl- test greylisting effectiveness. And then, admiring the result (but not the implementation) of qgreylistrbl, wrote greysmtpd .
Installing greysmtpd is very simple - it’s one of those that I “stuck and forgot”, you don’t even need to configure anything. The big plus of this approach is that spam instantly becomes about 85% less, and at the same time you are guaranteed not to lose a normal message. And if (when) spammers adapt to greylisting, then its effectiveness will drop, but there will still be no harm from it. In general, this is a must-have thing!
It should be noted that greysmtpd is optimized in order to minimize delays in delivering normal mail - in the vast majority of cases, normal mail will be delivered without any delay at all, as if greysmtpd were not used at all. This is achieved by using RBL and the built-in dynamic host identifier (DSL, dial-up, etc.): if the sender IP of the mail is not blocked in RBL and the sender domain (PTR) does not look like a dynamic host, mail from it is skipped immediately, without using greylisting. (Yes, I use RBL, but not to block mail, but just to force greylisting. For all of us, from time to time, we get to RBL.) In general, we get a kind of “soft greylisting” - a minimum of interference with normal operation at an excellent percentage blocked spam.
I tried to make the script itself reliable, simple and efficient. For example, if any error occurs in the script, it simply skips the current message, as if greysmtpd were not installed at all. SMTP Implementation lapped one-to-one with djb'shnoy utility rblsmtpd . And for efficiency, DNS queries are performed asynchronously. Well, plus all the possible optimizations that make it possible to avoid queries in RBL altogether, because RBL thing is braked. The script size is 200 lines, 5.5KB.
License, as usual, public domain.
Download: greysmtpd , deliver .
Criticism and code review are welcome. :)