How to do email-mailing and do not mess: practical tips

  • Tutorial

A developer who first encountered generating emails has almost no chance to write an application that will do this correctly. About 40% of letters generated by corporate applications have some kind of violation of standards, and, as a result, delivery and display problems. There are reasons for this: e-mail is technically much more complicated than the web, mail is governed by several hundred standards and an uncountable number of generally accepted (and not so much) practices, and mail clients are diverse and unpredictable. Testing can significantly improve the situation, but there is almost no material on testing mail.

Mail Mail.Ru regularly interacts with its users through emails. In our project, all components that are responsible for generating letters, and even single mailings are subject to mandatory testing. In this article we will share our experience (and stuffed cones).



What are the emails


The application can generate various types of letters. They can be classified into several categories. By the method of selecting recipients: trigger - selective - group. By appointment: transactional - marketing - service. You can set different requirements for different types of letters and apply different testing scenarios.

Trigger letters are generated in response to any events, for example, user actions or status changes of any system objects. They are generated by the application, and therefore the most interesting in terms of testing. Trigger letters can be both transactional and marketing.

Selective letters are sent to a dynamic selection of users that meet some criteria.

Group emails are sent to a known group of recipients, for example, all users or partners. Selective and group letters are often marketing, sending such letters is initiated manually or on a schedule.

Transaction letters are generated in the process of the user committing an action. Such letters include, for example, invoices, tickets or delivery status notifications, letters with an access recovery code, etc. Transactional emails are always trigger. Maximum compatibility is important for them, which means that they should be as simple as possible, and they need to be tested on a large number of clients.

Marketing letters encourage the user to take any action, for example, it may be an offer of individual discounts based on previous purchases. Transactional data can be used in these letters, they can be either trigger or mass - periodic or one-time. For these letters, efficiency is more important, usually it is determined by the results of the split test. Some aspects of compatibility can be sacrificed for efficiency.

Group marketing letters, for example, messages about seasonal offers, promotions and sales, are often sent “manually” and are not part of your application, but you can (and should) also apply general testing principles to them.

In addition, there may be service letters generated for employees for automated or automated CRM systems, journaling, auditing, or DWH. Such letters are triggered, which means that they are also part of the application and must be tested.

Who is involved in the testing and control process


  1. QA Engineer - participates at all stages of the process.
  2. Network Engineer - Responsible for configuring network infrastructure and message delivery infrastructure. Network engineer should be involved in planning infrastructure testing.
  3. A deliverability specialist is a person who monitors the deliverability of letters, who also participates in monitoring the technical and administrative parameters of all messages sent and monitors the progress of the mailing process. He is responsible for ensuring that the sent letters reach the maximum possible percentage of users and do not end up in spam, and for this purpose the specialist must have certain knowledge and contacts. If there are problems with the delivery of letters, it is he who must understand the probable cause and eliminate it: either by eliminating technical obstacles; or changing something in the content of the letters; or having solved the problem together with the support service of the mail provider, to which the letters do not reach. Such a specialist (if any) should also be involved in drawing up the checklist and testing the infrastructure generating applications and letters.
  4. Email-marketer - determines the effectiveness of marketing newsletters. Under his control, the split-testing for the distribution of marketing to the audience is required. Email-marketer also controls the segmentation of the user base, the composition and frequency of sent emails, the visual "presentation" of the letter to the user.

All these roles are not necessarily performed by a dedicated employee: the role of the marketer can be performed by one of the product managers, and the role of the deliverability specialist, for example, a support employee or a network engineer. And in startups, with a high degree of probability, all this will have to be dealt with by one person, and they may turn out to be a quality specialist.

Mailing and postal transport


The mail structure is like a huge iceberg, and there are two levels in it. There are more than a hundred different standards governing mail, but almost all of them belong to one of these two levels:

The underwater part of an iceberg- This is a network service whose base protocol is the SMTP application protocol defined by RFC 5321. It is responsible for the delivery of letters. For the delivery of the letter, a so-called SMTP envelope is formed, which includes the addresses of the sender and recipient of the SMTP level. Other network services, such as DNS, are also responsible for delivering the letter. The main component of the network infrastructure is the Message Transfer Agent (Mail Transfer Agent, or more commonly used abbreviation MTA). The MTA is responsible for handling the message delivery queue and the delivery process itself to the recipient servers. MTA examples are Postfix, Sendmail, exim, Microsoft SMTP service.

This underwater part of the iceberg, which includes the MTA, DNS parameters and other network parameters, we will call the mail infrastructure, or message delivery infrastructure.

The tip of the iceberg is the letter itself. The basic structure of a letter is defined by the standard RFC 5322. A letter consists of service headers and one or more data parts. The data may contain plain text text and / or HTML, inline images or attachments of almost any type.

The interface of the mail infrastructure and the boundaries of the application under test


The mail infrastructure, as a rule, has one or several interfaces through which an email is sent (enters the MTA delivery queue). For example, the SMTP Submission service, the functionmail()in PHP, transferring data to an external mail or sendmail application, internal or external services API (for example, GetResponse, SendPulse, or Amazon SES). We will consider these interfaces as part of the infrastructure. It often happens that application A prepares data for a letter and a list of recipients, and then sends it to application B through its API (for marketing mass mailings, this can be done manually via the user interface - UI), and application B generates a mail message in RFC 5322 and in some way puts it in the MTA delivery queue. From the point of view of application A (and when testing it), application B will be part of the mail infrastructure. The API or UI of application B will be for the application A interface of the mail infrastructure. Although from the point of view of Appendix B, the situation will be different

Determination of test parameters


When testing each application, it is important to select all the mail infrastructures used by it (there may be several of them), and for each infrastructure to single out the interfaces used (there may also be several of them for each infrastructure). For each interface, the composition and format of the data transmitted to it is determined as accurately as possible, for example: letter text in TEXT / HTML, letter text in TEXT / PLAIN, letter subject, recipient name, recipient address, sender name, sender address (RFC5321.From ), the address of the sender of the SMTP convector (RFC5322.mailfrom). Next, a set of requirements is developed for each parameter (representations, encodings, boundary values, etc.), methods for monitoring each of the parameters are determined (how to compare the actual result with the expected one).

Typical Generating Application Structure


For the generation of letters and data in it, as a rule, the same product that we are testing is responsible. This is usually a server (but sometimes client) application. It defines the structure of the letter, part of the service headers, data encapsulation formats, string representations and text encoding. A simple example of such an application is the script that forms the letter and calls the function mail().

The main elements of the application that must be controlled:

  • the code responsible for generating headers and / or letter structure, if the letter structure is dynamically generated, and / or a static letter pattern describing its structure;
  • HTML layout of the letter (ideally, it is a separate entity or part of the letter template / layout, but can be embedded in the application code);
  • substitution of application data into a letter (or into a letter template);
  • integration of the application with the letter delivery infrastructure, the correctness of the parameters passed to the infrastructure interface.

What and when to test


Whether we like it or not, the iceberg should be tested all. There are several main components that require testing:

1. Shipping infrastructure


Emphasis in testing should be done on: letter deliverability; correct DNS records, including PTR / FCrDNS, MX and A records; SMTP protocol parameters (HELO, use TLS); letter authorization (SPF / DKIM / DMARC); SMTP envelope addresses (if they are not managed by the application); correctness of processing the input parameters of the infrastructure interface; tracking, recording and processing of undelivered emails.

It is necessary to test the infrastructure during the initial implementation and every time changes are made to the infrastructure itself (the configuration of the MTA, DNS or network changes) or the interface for sending a letter; using a new domain, network, or API; The characteristics of the letters sent, such as their language, size or number, vary significantly. According to experience, the infrastructure tends to change without declaring war, so basic tests should be conducted periodically, even if there is no information about any changes.

It is possible and necessary to involve the network engineer and the deliverability-specialist in drawing up the plan and checklists for testing the infrastructure.

2. Generating application


The addresses of the SMTP envelope should be monitored (if they are controlled by the application, that is, they are transmitted to the interface — envelope-from, envelope-to), the values ​​of the service headers of the letter (Date, Message-ID, List-Unsubscribe, Auto-Submitted, etc.). Clause), letter authorization (DKIM / DMARC), MIME-encoding (base64, quoted-printable), general correctness of the letter format, for example, the absence of non-ASCII characters in the headers, the composition of the data to be substituted, the correct triggering of triggers, unsubscribe mechanisms, tracking mechanisms writing and collecting statistics (postmaster headers, for example, Feedback-ID or X-Mailru-Msgtype, as well as tracking pixels) .

It is necessary to test an application during its development, when all its related components that are responsible for generating and storing data change, with significant changes in letter templates, when changing the used infrastructure or its interface, as well as within the framework of general regressions.

3. Structural and inset letter templates (can be part of a generating application or are developed separately)


The structure of the letter is checked (Content-Type, Content-Disposition, nesting of Multipart-parts of the letter, text encoding, string parameters), the value of the target and displayed headers (From, To, Reply-to, Subject), the letter is displayed in the list of letters and when reading in various interfaces, microformats (for example, that a calendar event is recognized as a calendar event, or an air ticket as an air ticket), branding.

Letter templates should be tested every time you make at least the slightest changes, as well as separately, for example, in a situation where letters get into the application before the server part is ready.

It is recommended to involve an email marketing specialist and a deliverability specialist to compile a checklist for testing a letter template.

Basic requirements for checking infrastructure


The IP address selected for the mail server should be as close as possible to the IP address of the mail server. It is recommended to check it using the whois utility. In particular, the sender's address should not belong to the network, which can be perceived as dynamic; the selected network must have active contacts to which complaints can be sent; the network must be used (have the status ASSIGNED in RIPE). The IP address must have a properly configured PTR record. It can be configured independently through the hosting control panel, or with the help of the service provider. A PTR record must point to the real host name and still be meaningful, resolved back to the same IP address (so-called FCrDNS check), not remind the name of the dynamic host, and not include a large group of numbers or characters.

Each domain used in envelope addresses or letter headers must have a valid MX record pointing to a host A record, which, at a minimum, can handle undeliverable messages. MX is not allowed to directly refer to an IP address.

Control the passage of SPF, DKIM, DMARC. SPF allows the domain owner to specify in the TXT records a list of servers that have the right to send email messages with return addresses in this domain. It is configured for the address used in the envelope-from (SMTP-envelope) in the section of managing DNS zones of the domain. DKIM provides verification of the authorship of a message or of its originator to a specific domain using digital signature technologies, which is added to the letter itself (in its DKIM-Signature header). Typically, a DKIM signature is configured at the MTA (infrastructure) level. DMARC sets the policy of checking incoming mail in a specific domain and actions on letters that do not pass SPF or DKIM authentication. When attempting to violate this policy, a structured report comes along with information about such an attempt. DMARC, as well as SPF,

Check the deliverability of letters to the main postal services - for Russia Mail.Ru, Yandex, GMail, Microsoft (Hotmail / Outlook.com / Office365), Rambler, nic.ru. In the letters arrived, you need to check the correctness of HELO; the presence and passage of checks PTR / FCrDNS, SPF, DKIM and DMARC; the validity of the headers and data in them, in particular, the synchronism of the clock in the dates and the correctness of the time zones.



The formation of some parameters, for example, authorization, deliverability, and spam, are integrally influenced by all components, but for their control there are usually separate operational tools — DMARC and FBL reports, postmaster services API, letter tracking tools, delivery statistics. Testing should take into account the level of implementation of operational monitoring tools in the company - for example, in the absence of operational control of DMARC reports, the authorization of letters should be regularly tested, in the absence of operational control of deliverability - to regularly check how and where letters get, even if there is no development related to sending letters is not conducted.

To test the infrastructure, you can use specialized services, for example, mail-tester.com ,mxtoolbox.com . Detailed infrastructure requirements can be found in this article .



Authorization Requirements


Check the passage of SPF, DKIM, DMARC can usually be on the Authentication-Results header on the recipient server.

Check the validity of the SPF record for syntax, limit for DNS queries (for example, using mxtoolbox.com). When forming SPF, all sources of mailings should be taken into account (do not forget CRM systems, all delivery infrastructures used, including those through which one-time marketing campaigns are conducted). It is recommended to specify allowed servers for the domain through the list of networks ('ip4' / 'ip6'). SPF is checked for the sender address from the SMTP envelope. Verify that the domain of the SMTP envelope (envelope-from) matches the domain from the From header. Typical SPF errors are listed in this article .

Check the DKIM DNS record, the validity and composition of the DKIM signature (DKIM-Signature). Check that the DKIM key is at least 1024 bits in size. The recommended DKIM signature hash mode is relaxed / relaxed. Make sure that all important headers are signed (From, To, Subject, Date, Message-ID, MIME-Version, Content-Type), tracking headers are not signed (Received, Delivered-To, Return-Path), and DKIM is validated on main postal services. Set up one of the postal services forwarding to another, DKIM should not "beat" on the forwarded letters. Ensure that the DKIM signature domain matches the sender's domain from the From header.

Check out the progress of the DMARC on the main postal services. Check for DMARC reports, identify and fix problems with SPF and DKIM for all IP addresses of your infrastructure.

Verify that letters are delivered to external servers using encryption (TLS). It is sometimes possible to check for TLS by the Received header on the recipient's server: for example, specifying the ESMTPS protocol or the presence of parameters of the form (version = TLS1_2 cipher = ECDHE-RSA-AES128-GCM-SHA256 bits = 128/128); indicates the presence of TLS.

Verification of the generating application


Requirements for postal addresses


Envelope Addresses We
begin the verification of the generating application with addresses in the SMTP envelope.

Envelope addresses are addresses of the level of postal infrastructure. They are not visible to the user, but are important for delivery, because the mail box in which box gets is determined by the envelope address.

The recipient's address in the envelope (envelope-to, aka RCPT TO :) is the address to which the letter will actually be delivered.

  • for all letters, except for registration, the address must be legally signed and validated for distribution in accordance with administrative requirements.
  • for mailings, the address must be “live”, addresses to which it is regularly impossible to deliver letters should be marked as inactive, mailings should not be made to them. But some categories of letters (for example, restoration of access) may need to be sent to addresses previously marked as inactive.

The sender's address in the SMTP envelope (usually called envelope-from, smtp.mailfrom or Return-Path) - messages on the inability to deliver the letter and auto-replies will be delivered to this address. The user is not visible to this address. This address is also used to authorize SPF. Check that:

  • the address is available;
  • It is not an employee’s address and is not redirected to it in order to exclude auto replies, inaccessibility messages, etc .;
  • processed by a script that will mark the addresses of inaccessible users as inactive;
  • the address can be automatically generated, i.e. unique to each letter;
  • the letter to this address should not lead to the generation of any response letter, for example, a message about the overflow of the box.

Mail Header Addresses

These addresses are either directly visible to the user, or used when replying to a letter.

The sender's address (From :) header is the address and the name of the sender displayed in the list of letters and when reading a letter. Check that:

  • This is a “human readable” friendly address that identifies the company.
  • Contains not only e-mail, but also the name of the sender.
  • noreply @ is possible, but only if we want to emphasize that we do not expect to receive an answer to the letter and it will not be read. It is better to duplicate this idea in the text of the letter.
  • If there are non-ASCII characters (for example, Cyrillic), the sender's name must be encoded according to MIME, the domain if there are non-ASCII characters must be encoded in Punycode.
  • Letters of the same category should have the same address; the use of auto-generated addresses is not allowed. This is due to the fact that From: is most often used by people to sort letters into folders using filters.
  • The address should be different (better in different subdomains) for transactional, marketing and urgent letters (such as letters from the support service). This is also due to the fact that the user can mark marketing letters as spam or filter them into an unreadable folder.
  • The From address and the SMTP envelope address must be in the same domain or in subdomains of the same organizational domain for passing SPF within DMARC.
  • The address must be from the organization's domain. It is unacceptable to use the From domains of free mail services and other public mails, since such mailings will not pass SPF and DKIM authentication within DMARC.

Address Reply-To - the “manual” answers will be sent to this address when the user answers the letter. It is not obligatory: in its absence, the address from From is used for the response. Check that Reply-To:

  • It can be autogenerated, i.e. unique to the letter (allows you to know which letter came to answer).
  • It should not fall into the employee's box to avoid auto-answers, and ideally should be “wrapped” in CRM.
  • It can generate a standard CRM auto-reply, but should not generate anything superfluous, for example, messages about a box overflow. When generating auto-responses, measures should be taken to avoid looping; they are listed in RFC 3834.
  • It may be from any domain that does not necessarily coincide with From, but sometimes it scares users when they answer.
  • May be absent, then its function is performed by the From address.
  • In addition to the address indicated the name of the sender.

To address:

  • Must contain the recipient's e-mail (otherwise it scares the recipient of the letter and the antispam alarming).
  • Ideally, should contain the name of the recipient. But if the name is unknown or doubtful (for example, the address has not yet been confirmed), it is better not to indicate it (someone may enter someone else’s address with a bad name, and the recipient might be offended by you).

Mail Header Requirements


The actual encoding of the text must match the one specified in the title. It is advisable to use the same encoding in all headers and parts of the letter. It is recommended to use UTF-8 as widely supported. The encoding is specified in the Content-Type headers and in the <MЕTA> tag of the HTML part.

The headers From :, Message-ID: and Date: must be formed directly in the script to send the letter (and by standards - along with the text of the letter) and always in the correct format. In the case of their absence or incorrect formation, one of the transit servers can add these headers, which leads to a violation of the integrity of the DKIM signature.

8-bit characters in the headers, including the subject line (Subject) and the names of the attached files (Content-Type and Content-Disposition), should be absent; all non-ASCII characters, including Cyrillic, must be encoded in quoted-printable or base64.



Requirements for the structure of the letter


For the HTML part of the letter, it is desirable to form an alternative - text (plain) part. It is also necessary to check the conformity and readability of the plain text-part of the letter (if available) and the general structure of the letter.

According to RFC 5322, the length of a string in a letter should not exceed 998 8-bit characters. Note that in UTF-8, a character may take several octets. The string terminator in the email is the pair CRLF (<cr> ascii 13, lf> ascii 10), which occupies 2 octets. It is necessary to check the correctness of the string terminator, since letters are often sent using a Unix script, and on Unix, a single character lf is used as a string terminator - for email, this is an error. You should also check whether the terminators break strings of characters in the UTF-8 encoding: you cannot assume that there is a terminator between two octets of one character, for example, Cyrillic. To avoid such situations, it is necessary to break the text before forming the letter, or encode the text in base64.

It is necessary to check the correctness of the markup of attachments and inlines - that is, the correctness of the formation of the "Content-Disposition: inline" headers, if this is a picture displayed inside the letter, or "Content-Disposition: attachment", if the file is intended for downloading.
The letter structure, if possible, should be as simple as possible: in particular, there should not be more than one multipart-part of each type (mixed, alternative, related), and multipart / mixed is used only if the letter contains attached files, multipart / related - if available inline images, multipart / alternative - in the presence of plain text- and HTML-parts. In general, the structure of the letter, in the absence of attachments and inline pictures, should look like this:

multipart / alternative
- text / plain
- text / html

The order of the parts is important, text / plain must go BEFORE text / html or multipart / related. This is necessary so that the HTML part is displayed by default, and only if its display is unavailable for some reason — the plain part will be displayed.

If there are inline-pictures in the letter, its structure should look like this:

multipart / alternative
- text / plain
- multipart / related
—— text / html
—— image / ... (inline picture)
—— image / ... (inline picture)

Inline pictures must have Content-Disposition: inline and be strictly inside the multipart / related-parts.

In the most difficult case, when there are both inline images and attached files, the letter has the following structure:

multipart / mixed
- multipart / alternative
—— text / plain
—— multipart / related
——— text / html
——— image / png
——— image / png
...
- application / octet-string (content-disposition: attachment)
- application / octet -string (content-disposition: attachment)
...

(multipart / related- and multipart / alternative-parts must be closed before attachments, attachments belong to the external multipart / mixed-part)





URI requirements


Any URIs (in src, href, styles, etc.) must contain the protocol and host name ( https://example.com/somepath ). Typical errors are the use of relative links (/ somepath) and the absence of a protocol (//example.com/somepath), which is unacceptable for letters, since in them, the default protocol can be file: //.



  • Any service and nonASCII characters (in particular, Cyrillic) in the URI must be encoded with percent encoding.
  • A link inserted as text (that is, visible to the user as a URL, and not as a fragment of text) must still be tagged through the <a> tag, otherwise the user will not be able to click on it. Some webmails tag these links on their own, but this is not standard behavior. The href address inside A in this case must match the text of the link, otherwise the content filter may react to this link as an attempt to deceive the user. You should especially pay attention to this if there are “clickers” that track the user's transitions from the letter.
  • It is better to restrict the use of protocols http: //, https: // and mailto :.
  • With high security requirements, you should completely abandon the use of http: // in favor of https: //.
  • Non-standard ports should not be used (for example, example.com : 8080 / somepath), because they may not be available to the user.
  • Clicking on the link inside the HTML part should not lead to any changes in the application state (subscription, unsubscribe, order cancellation, etc.) without additional confirmation by the user on the page, since Some content filtering systems can independently verify the safety of such a transition by requesting a page by reference; the mail application can show the page preview by reference when the mouse is over, and modern browsers can load the page before the user clicks the link to reduce the download time (in a web application, it is not recommended to do any modifying actions on the GET request, all modifying requests must go through POST or PUT).
  • Follow the link in the header of the List-Unsubscribe, on the contrary, should not require any additional actions from the user, because for the user, the mailer usually replies to this header.
  • You should not expect the user to read the letter and follow the link in the same browser in which he initiates the action leading to sending the letter (for example, registering an account). The link should work in any other browser or mobile device. In particular, a user can open a link, being unauthorized, or authorized in an account other than the one to which the letter was sent.
  • Because URI length can be limited, it is not necessary to use data: URI for large objects. For the same reason, do not use too long URIs in the links.
  • You cannot use external reference abbreviations, this negatively affects the deliverability of letters. It is better if all links point to your domain, it will reduce the potential negative impact of someone else's reputation on the deliverability of letters.
  • You should not have external images on some public services or free hosting.

Letter Layout Requirements


Why is it so difficult to make up letters?

E-mail clients somehow show user-generated content within their interface. This can potentially lead to various security problems - cross-site scripting (XSS, Crossite scripting), interface spoofing, DOM clobbering, user de-anonymization / information leakage (for example, user's IP addresses or cookies through external requests), etc. ., therefore, any mail service and mail application has a certain degree of protection against each class of attacks. Unfortunately, there is no single approach to the organization of this protection. It can be organized through:

  • isolated limited frames
  • filtering tags and / or attributes
  • restrictions on absolute positioning,
  • prohibition or restriction on the use of block styles (which is critical for adaptive layout),
  • prohibiting external elements by default (i.e. loading external images requires user permission) or using a proxy to access them,
  • converting HTML emails into another intermediate format (for example, Microsoft Exchange / Outlook uses RTF, which makes it possible to achieve adequate display of items in Outlook using conventional methods),
  • prohibition or restriction on the use of forms or their individual elements.

Also, letters use specific elements, such as inline images and the cid: // URI, whose support may be limited. For example, Mozilla Thunderbird does not support cid: // for background images.

Even a well-formed letter can be displayed differently in different interfaces due to the peculiarities of their implementation and filtering the contents of the letter.

If there are errors in the letter format, the behavior becomes completely unpredictable. For example, in email clients there may be different behavior with incorrect URIs, incorrect header formatting is handled differently, encoding autodetection works differently if it is not specified or is incorrectly specified. Therefore, it is necessary to look at the letter in different interfaces: the correct display of the letter in one interface does not mean that it is composed correctly (in fact, even a correct display of the letter in all interfaces does not guarantee the absence of problems with the display in the future).

It is necessary to pay attention to the following points:

  • Check the text of the letter for semantic content, display, no typos, syntax, spelling and lexical errors.
  • Check the correctness of the application data substitution in the letter template or layout.
  • Check the correctness of amounts, dates, numbers, positions of goods and other information, taking into account the permissible boundary conditions. The dates must contain a year (some users go to the box very rarely). There must be a time zone in time. The address must contain a city, and in some cases it is necessary to indicate the country.
  • Check the working status of all links in the letter, if any.
  • In the letters sent before the confirmation of the address, including confirmation emails should not contain any text controlled outside, even the username, otherwise they can be used for spam (in the field displayed in the email, for example, in the name, spam text is inserted and the address is the address of the victim). For example, if you can, on behalf of your service, send obscene text to the address of a developer, then there is a problem.
  • Check the absence of external images on third-party services.
  • Check the availability of counters sending, delivery, reading the letter, transitions. Some of them are in the letter itself (for example, a counter-pixel reading of the letter), some are tracked by the sender, but, as a rule, all are available in the admin of the sender.
  • Check the correctness of the subscription category and the work of unsubscribe the user for this category through the link in the letter
  • Check display in:
    • popular web versions of mail: Rambler and Outlook.com can be added to the “big three” Mail.Ru, Yandex, Gmail;
    • mobile applications of the above mail providers;
    • standard mobile applications using the IMAP protocol, taking into account popular mobile platforms, at least for iPhone, Pixel (Android reference platform), Samsung (most common for Android), MIUI (second place for Android platforms);
    • various desktop browsers: Chrome, Firefox, Edge, Internet Explorer, Opera, etc .;
    • desktop applications (email programs), surely Thunderbird, Outlook and Apple Mail, optionally The Bat! and Opera Mail;
    • popular corporate solutions with a web interface (Exchange, optional Roundcube, Communigate, Zimbra, SquirrelMail) - for B2B solutions;
    • Do not forget to check the layout on both Retina-monitors and on monitors with a lower resolution.
  • During the test in each case, you need to pay attention to:
    • Passage of authorization headers, SPF / DKIM / DMARC.
    • Letter loading speed: it should load quickly, not hang.
    • Displays the letter in the list of letters: the avatar, the sender's name and the topic that falls into the snippet of the letter, whether its category was correctly determined (for example, whether the order fell into the “social network” category).
    • The layout of the letter as a whole: whether nothing has parted, whether there are any incorrect hyphenation of words, etc., including when scaling and resizing the window.
    • Fonts should not be small or poorly readable.
    • Background images and background colors.
    • Compliance brandbook.
    • The convenience of performing the actions implied by the letter. For example, if a letter contains a confirmation code or other information that you may need to save somewhere, then it should not only be read well, it should also be convenient to select and copy it even in the mobile interface.



  • Watch for the overall size of the letter (including external images) and that it does not exceed reasonable values. The harder the letter for traffic and download time, the more likely a negative reaction to it.
  • Even letters that are not modified should be periodically checked, because changes can occur on the postal service side, and, for example, a previously invisible problem can “shoot”.
  • Some parameters need to be controlled in all tests. For example, problems with the passage of DKIM authentication may be due to infrastructure problems (problems with DNS or DKIM signature generation, time synchronization errors), due to software errors (the sender's address is incorrectly formed, incorrect characters in the headers, are skipped or mandatory headers From, Date or Message-ID are duplicated and because of content errors (incorrect line terminators, too long lines, incorrectly specified addresses). At the same time, a letter may not “beat” everywhere, and the problem may not appear on any service.

Split tests


Marketing research is beyond the scope of this article, but several main points that significantly affect the quality of letters should be mentioned.

The newsletter has a goal, so it must take quality, not quantity (as spammers do). Mailing must be segmented. When carrying out an advertising campaign, you need to know exactly who is in the segment sample, why he offered the product and what you want to convey.

For each newsletter, it is necessary to calculate the CTR of the list of letters - this is the ratio of the number of read letters to the total number sent. In postmaster.mail.ru you can see the rates for unique users. If the measurement goes through the counter in the letter (pixel), then the absolute number of discoveries is estimated. CTR <10% is a very low indicator, it is undesirable to conduct such a newsletter. It should aim for a CTR> 30%. For marketing letters, the CTR of clicking on the links in the letter and the percentage of actions performed (“sales”) on these links are also very important. Be sure to follow the complaints (marking the letter as spam). Usually for one-time mailings, a tenth of a percent is a good indicator, for regular ones - a hundredth of a percent. The critical values, after which the mailing is practically treated as spam, are indicated here:https://help.mail.ru/developers/mailing_rules/technical .

It is necessary to conduct split-testing of various distribution options for obtaining optimal indicators. Just changing the name of the sender and the subject of the letter can increase the CTR by several times and repeatedly reduce the number of complaints. The number of letters should be statistically significant for evaluating the results (for large projects it is usually several thousand). The final version of the letter is sent in several stages for additional measurement of indicators and “warming up” - starting with approximately 10,000 recipients, with an increase of about an order of magnitude per day.

Main idea: emails are part of your application, perhaps one of the most complex and problematic. At the same time, this is often a “blind spot” in terms of testing. I hope that I managed to draw your attention to this problem.

I express my deep gratitude for the help in the preparation of the article to Vladimir Dubrovin z3apa3a and Alena Likhacheva s4ever . The article also used the materials of Edward Tyantov EdT and Alexander Purtov 4Alexander .
Tags:

Also popular now: