There is quite a bit of information about such integration on the network. The fact that the problem is solved is evidenced by rare comments on the sites, but I did not find a ready-made solution. Perhaps this is due to the fact that most use postfix / exim.

Objectives of the article:

1. Describe the methodology for solving the problem, so that it is easier for those who follow my footsteps;
2. Distract the community from the good old postfix and company.

Introduction

In the process of moving the finished web / mail / server infrastructure from the old server to new servers, I was faced with the need to deploy the mail infrastructure again.

The original mail system was deployed in 2012. Ansible was only in its infancy. Configuration systems were not as common as they are now. I then loved automation of work less than now, so everything was configured by hand.

The new system was supposed to be built on the docker (which means the most detailed description of building the system "for the future") and work in the docker cluster.

The configs of the old system were preserved, but I really didn’t want to return to “dnl” sendmail, a bunch of description lines for postfix logic, an incomprehensible process for setting up amavis + spamasassin, a lot of memory that required amavis + spamasassin. It was decided to look for alternatives.

So that no one says "use some kind of marginal software because you have not mastered postfix", I will clarify: mastered. The original system consisted of postfix + dovecot + amavisd-new + spamasassin + dkim-proxy + razor + pyzord + dcc.

Looking ahead: the final system consists of opensmtpd + dbmail + dkim-proxy + rspamd.

Bit of theory

opensmtpd is a lightweight mail daemon with an elementary configuration process from the OpenBSD team. The configuration rules are so simple and intuitive that they are immediately remembered and allow you to describe in ten lines the complete logic of the mail server.

dbmail - mail storage with pop / imap interface. The main feature is the storage of mail in the database, which allows you to create an arbitrary number of entry points.

rspamd is an anti-spam mail system from developers from Rambler.

More alternatives? Why not Zimbra? Why not some ready-made mail-in-a-box solution that contains all the necessary components?

Zimbra fell away due to Java - I did not want to spend resources.

mail-in-a-box - again, all the same postfix and company.

Description of the problem

opensmtpd is a great daemon, but it has flaws that prevent it from being easily integrated with rspamd.

It:

1. Lack of support for the milter protocol. This protocol (extension) would allow integration with rspamd using standard tools.

   
   

2. Lack of filter support. The developer of rspamd for integration recommends using opensmtpd filters. The development of filters has been going on for many years, but is still not in production. There is even a filter for rspamd, but still in a separate thread on github.

   
   

   After certain modifications to the code, I was able to compile and run this filter, but it threw errors during the work and there was no desire to understand the code further.

   
   
3. The lack of a method of receiving mail other than by mail protocols.

Also, the situation was complicated by the following points:

1. Remoteness of services from each other: daemons work in docker containers with different IPs. In fact, a container with opensmtpd and a container with rspamd can be thought of as running on different servers.

   
   
2. Software heterogeneity: opensmtpd is in the stable version of Alpine Linux (the linux version that is often used as the base Docker image), but rspamd is not there. And vice versa: rspamd is in the unstable version of Alpine, but opensmtpd does not work there.

So, as an initial task, we have opensmtpd, which on port 25 is waiting for incoming mail and on port 10029 is waiting for mail that rspamd checked.

Solution

In opensmtpd there are mda. This probably stands for mail delivery agent. Redirecting mail in mda causes an external program to start. An example of a redirect rule:

accept tagged IN from any for domain  userbase  deliver to mda "/usr/local/bin/rspamd.sh %{sender} %{rcpt}" as admin

Here we call some external script where we pass the sender address and the recipient address.

Included with rspamd is rspamc, which is an analogue of the mail mta for rspamd. This program allows you to accept a letter as an input, transfer it to rspamd via the HTTP API and issue a response.

The integration of mda and rspamd is as follows

/usr/bin/rspamc --mime -h rspamd.example.ru:11333

We send an email to the STDIN script, specify the rspamd address and say with the --mime parameter that we need to add the message processing headers (which will be used to determine spam).

After the message has been processed by rspamd, it needs to be sent back to opensmtpd somehow. But the standard mta (complete with opensmtpd) can only send to port 25 (we are waiting for a verified letter on another port).

Solution: create communication by the mail protocol ourselves (well, it is simple) and send data to the desired port using netcat.

Final mda rspamd.sh script for integration of opensmtpd and rspamd

#!/usr/bin/env sh
mail_file=$(mktemp)
echo 'HELO localhost' >> $mail_file
echo "MAIL FROM: <$1>" >> $mail_file
echo "RCPT TO: <$2>" >> $mail_file
echo 'DATA' >> $mail_file
/usr/bin/rspamc --mime -h rspamd.example.ru:11333 >> $mail_file
echo '' >> $mail_file
echo '.' >> $mail_file
echo 'QUIT' >> $mail_file
cut_file=$(mktemp)
sed '/Delivered-To/d' $mail_file > $cut_file
rm "$mail_file"
count=0; IFS=''; while read -r line ; do if [ "$count" -gt "5" ]; then sleep 0.05; else sleep 0.1; fi; echo "$line"; count=$((count+1)); done < "$cut_file" | netcat localhost 10029
rm "$cut_file"

Features of this script:

1. Data should be sent via netcat with delays after each line of the connection setup phase. To do this, we make delays between the return lines.
2. From the original letter, the "Delivered-To" header, which opensmtpd adds, should be cut. Otherwise, opensmtpd will consider this letter to be looped.

Solving the problem of combining opensmtpd and rspamd in one container: using a Docker image based on the Debian sid, where the rspamd repository is connected and the latest version is installed from it. Installation example:

wget -O- https://rspamd.com/apt-stable/gpg.key | apt-key add - \
&& echo "deb http://rspamd.com/apt-stable/ sid main" > /etc/apt/sources.list.d/rspamd.list \
&& apt-get update \
&& apt-get install --no-install-recommends --no-install-suggests -y \
rspamd

Summary

As a result of the transition from postfix + amavis to opensmtpd + rspamd, I got the simplicity of the service configuration, low memory consumption and less spam for mail.

The implementation of the final Docker container can be seen here .