HOWTO setup ubuntu 18.04 sendmail + DKIM + SPF + DMARC or how I managed to get into the Gmail bastion

Foreword

There are 2 separate domains on my server. Prior to the Roskomnadzor blocking history, we collected traffic of about 2,000 visits per day, and the mail server sent about 200 emails per day to all popular mail services, incl. Google and Yandex. Everything was great. But as they say in the famous video: “Everything was so good until Navalny Roskomnadzor came !

Now, when we managed to find a hoster, whose IP addresses do not fall under the state censorship blocking, another problem has appeared. Google Good Corporation has ceased to let our postman on the threshold.

The google MX server throws out my mail with something like the following message: “Your message looks like an undesirable one, bye bye ...”


I sent a test message with this command:

echo "Subject: Hello baby!" | sendmail -v legale.legale@gmail.com

I will not write about my dances with a tambourine in an attempt to break through the spam filters. I can only say that I was taken all night, but I could only succeed the next day. That is why I decided to write this manual.

INSTRUCTIONS FOR SETUP SPF + DKIM ON UBUNTU 18.04

The steepness of this manual is that it works, unlike many others. I guarantee that you can configure at least 2 domains on 1 server without much difficulty.

1. Install the necessary packages

apt-get install sendmail opendkim -y

2. Configure opendkim

The config is here: /etc/opendkim.conf

3. Configure keys and signing rules

I will describe the procedure for creating your own keys for those who want complete autonomy. Personally, I use the Yandex Mail domain service, pdd.yandex.ru, so I have the keys generated by Yandex.

#сначала делаем каталог
mkdir -p /etc/opendkim/keys/*****.ru
#переходим туда
cd /etc/opendkim/keys/******.ru
#делаем ключи
#-s определяет селектор -d домен для которого делать ключ
opendkim-genkey -s mail -d ******.ru
#должны появится 2 файла mail.txt с открытым ключом и mail.private с закрытым
#назначим владельцем файлов ключей opendkim:opendkim
chown opendkim:opendkim mail.*
#ограничим доступ к секретному ключу
chmod 600 mail.private

Next, we connect our DKIM secret key created or uploaded from Yandex.

We are interested in these three lines from the opendkim config:

InternalHosts refile:/etc/opendkim/TrustedHosts
KeyTable refile:/etc/opendkim/KeyTable
SigningTable refile:/etc/opendkim/SigningTable

The first file /etc/opendkim/TrustedHostsstores the hosts that will have access to the opendkim server for signing letters.


The second file /etc/opendkim/KeyTablecontains a table of secret keys and related DKIM records, which are the following:[selector]._domainkey.[domain_name]


In the third file /etc/opendkim/SigningTableis a table of the rules of signing letters. Here it is indicated whose letters and with what key to sign.


We create these files by specifying your domain and your path to the key file. This completes the opendkim setting.

4. Configure sendmail

Configuring sendmail is as simple as possible. We just need to add the /etc/mail/sendmail.mcfollowing lines to the end of the config file prototype file :

#Эта строка включает поддержу starttls
include(`/etc/mail/tls/starttls.m4')dnl
#Заставим sendmail пропускать письма через opendkim
INPUT_MAIL_FILTER(`opendkim', `S=inet:8891@localhost')dnl

For the default mail to be sent from the correct domain, you need to make another change in the file. /etc/mail/sendmail.mc

We are looking for a line that looks like this: MASQUERADE_AS(`st-goods.ru')dnlReplace with the required domain. This domain will be used as the default return address. Notice that the quotes are different there; strictly speaking, the first character `is called a gravis. So for some reason it’s got in the sendmail config.

Now we will add entries to the / etc / hosts file. This is necessary in order for sendmail to pass letters through the filter. It has been empirically established that if no entries are added, all logs remain pristine clean, not showing a single error, but the letters are not signed.

#ipv4
echo -e "127.0.0.1 st-goods.ru" >> /etc/hosts
#ipv6
echo -e "::1 st-goods.ru" >> /etc/hosts

Now you need to rebuild sendmail with new settings.

#запускаем скрипт 
#на все вопросы отвечаем "да"
sendmailconfig
#перезапускам службы opendkim и sendmail
service opendkim restart && service sendmail restart

Now our sendmail server is able to pass outgoing emails through the opendkim server, which adds a digital signature and encrypts the headers of the sent emails. It remains to publish the public key for the receiving party so that you can decrypt the headers and make sure that the headers are not changed.

5. Settings DKIM TXT records on DNS server

The key must be hung on the DNS server in the TXT type string as follows:
host: mail._domainkey
value: v=DKIM1; t=s; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC2wtGTw/5KPjtlIEh282JY7ovxZ/8eqveFn9ivhzpYJldl3fBEOKw

Explanations:
p=this is our public key that
v=determines the version number of DKIM
t=sets the flags, by default the flags are not set. There are 2 flags 'y' and 's'.
ysays that DKIM works in test mode.
smeans the mode when the right side of the email address after @ must completely coincide with the domain specified in the d = DKIM parameter of the signature of the letter.

The host must be specified as follows: [selector]._domainkey
We have the mail selector, so the host will be like this: The mail._domainkey

public key must be removed from the file/etc/opendkim/keys/st-goods.ru/mail.txt


The key goes from p = to the last quote. The quotes with spaces must be removed.

6. Configure SPF

SPF (Sender Policy Frameword) is another 1 anti-spam technology, I will not talk about it. Only 2 words to understand the essence of this technology. The DNS records of the domain are similar to the DKIM public key. The addresses of servers that have the right to send mail from this domain are recorded. We need to register in this entry ip address of your server. In my case, a Yandex server has also been added to the list.

Here is an example of my record:
host: @
value:v=spf1 a mx include:_spf.yandex.net ip4:173.249.51.13 ip6:2a02:c207:2018:3546::1 ~all

The syntax is simple. First, the spf version is specified, and then the spaces of the servers that have the right to send mail from this domain are indicated through a space with or without a plus. I have a and mx records, which means that the servers from DNS A and MX records are also on the allowed list. Via include: the address of the Yandex server is specified, and then the ipv4 and ipv6 addresses of my server. ~ all means soft refusal. Those. all addresses that are not listed in the resolved can still pass at the discretion of the receiving server. -all would mean a hard failure.

7. Configure DMARC

The last nail to penetrate the gmail bastion is the DMARC. DMARC sets what to do with emails that do not pass SPF or DKIM.

We do everything just as well through a DNS TXT record.
host: _dmarc
value: v=DMARC1; p=none

Here we establish that if the letters did not pass SPF or DKIM, then nothing needs to be done. You can put p = reject. Then such letters will be rejected.

Check

We send a letter. echo "Subject: Hello baby!" | sendmail -v legale.legale@gmail.com

We look at the last records of the mail log. tail -f /var/log/mail.log | grep dkim
A line similar to this should appear:

Jun 11 22:07:55 sevenlight opendkim[6473]: w5BK7sl9008069: DKIM-Signature field added (s=mail, d=st-goods.ru)

If there is a string, then the sendmail server and opendkim worked together and signed your letter. If there is no such line, see the “Possible Problems” section.

Now we look what happened in the mailbox. Open the letter and press the button with the down arrow in the upper right corner of the letter. In the drop-down menu, select the item “Show original”.

That's what happened with me:

Message ID <201806112007.w5BK7sUS008068@******.ru>
Created on: 11 June 2018 at 22:07 (Delivered after 2 seconds)
From: info@*****.ru
To: legale.legale@gmail.com
Subject: Заказ №2221 Интернет-магазин напольных покрытий ******.ru
SPF: PASS with IP 2a02:c207:2018:3546:0:0:0:1 Learn more
DKIM: 'PASS' with domain *****.ru Learn more
DMARC: 'PASS' Learn more

PROFIT $$$

Possible problems

I sincerely hope that there will be no need to read this section, but the technique is a difficult thing ... Sometimes it is not clear who works for whom.

First, we check everything you did in the exact sequence specified in the instructions.
First, we check the / etc / hosts file for correctness of changes in it. I had the most problems there. Next, check the system log for errors dkim. While I was picking, I happened to face the situation of the impossibility of reading the secret key opendkim, although I appointed the user opendkim as the owner of the file. Then you should carefully study the sendmailconfig command execution log. I had a case in which I used a regular single quote instead of a fixed gravis, and sendmailconfig cursed it. If nothing helped write me and keep trying.