How not to turn off pictures and scripts in letters, and at the same time be safe. History from Yandex.Mail

    Today I want to talk about how we made it so that Yandex.Mail didn’t have to turn on the display of pictures in each letter, how to do this a lot where, and in general - how we provide protection when displaying the text of the letter. That is not such an obvious thing as it may seem.

    Our users receive about one hundred million letters a day and read about thirty million through the web interface. This is a huge field of activity for attackers, so we built a multi-level defense. Messages are scanned for spam, phishing, viruses, malicious content and links. Under the cut, we will tell you about the protective mechanisms through which letters pass before being displayed in the web user interface.

    Spam defense

    More than 90% of all emails on the modern Internet are advertising or malicious spam. Spam defense effectively suppresses attempts to send spam, including unauthorized mailings that not only offer you to buy something, but try to actively attack you personally or your computer. There are several attack vectors. The most common are phishing and mailing malware: viruses, trojans, bots.

    Spamooborona stops 50-70 thousand phishing emails daily. Phishers try to attack both Yandex accounts directly and use Yandex addresses to try to steal accounts on other services. Interestingly, attacks on accounts in online games and game stores such as World of Tanks or Steam are very common.

    We analyze the texts of letters, and also use technologies such as DMARC to identify and combat phishing emails.


    The next level of protection is the good old antivirus. In 2014, real live computer viruses in the form of executable files are emailed quite rarely. We see no more than several thousand such letters per day. Nevertheless, each such file is potentially very dangerous, therefore it is vital for us to prevent infection of our users' computers. For many years, Dr. Web, whose server-side antivirus we run on a fairly large cluster of thirty machines in full scan mode of all incoming emails.

    The second phase of antiphishing

    Spam defense filters letters at the moment when they arrive at our incoming mail receiving server (MX). An important and understandable drawback of such a regime is that sometimes a threat becomes widespread, and good signs for identifying it appear in us after a certain number of such malicious letters have been received in user boxes. For such situations, we have one more level of checks, which is performed at the moment each specific letter is displayed in the Yandex.Mail web interface.

    Immediately before sending a message to the user, his text is scanned to match a short list of lines and regular expressions. An important property of this list is that it can be edited instantly, which is very important if an attack is detected that is happening right now. Every second, Yandex.Mail accepts up to several tens of thousands of letters and delays inevitably associated with updating large databases are very harmful here.


    Modern email is HTML. No matter how we, champions and guardians of purity of standards, do not resist progress, users make their choice. Alas, this also means that all the richness of expressive means that is now available in HTML is wrapped up by the need to carefully scan the markup and prevent attempts to use "active" elements to attack webmail users. HTML was developed without taking into account situations when one document is safely embedded inside another, and this is exactly what you need to do when displaying HTML letters in the web interface (that is, in fact, inside another HTML page). We simply called this component a sanitizer. It parses HTML at the character level, not object elements, since many attacks on web interfaces use markup that is not valid in terms of language standards to circumvent the simplest checks. Now HTML is not just one language, but a whole family, and the sanitizer separately knows how to parse the CSS description language inside certain elements and attributes. The result of the sanitizer’s work is a simplified text of the letter that you can safely paste into another HTML page and not be afraid that some script will suddenly run that the whole DOM of the web interface will be accessible or that the styles from this internal block will suddenly affect the elements behind it the limits.

    Display letter

    For quite some time, the Yandex.Mail web interface only works using the secure HTTPS protocol. In addition to fully encrypting traffic between our servers and the user's browser, we also use an additional secure cookie, which is not available without encryption, to authenticate requests. Thanks to her, even if your provider stole your authorization while you were without encryption, but in a logged state, watched the Yandex Poster or simply visited any site using the Yandex.Metrica counter, it could not get into your inbox. To do this, he will need an additional cookie, which can only be obtained by decrypting the HTTPS traffic.

    Check for viral links with unwinding shortened links

    We do not stop there and continue to protect our users even after the letter is already displayed on the screen and can be read. The most dangerous letters are trying to get the user out of the secure Yandex.Mail interface out and already there to do something bad with him. Therefore, at the moment of clicking on any external link in the letter, another step is triggered - checking the link in the database of malicious links of a large Yandex web antivirus. We already wrote a littleabout this earlier, but it will be appropriate to repeat that our web antivirus constantly “indexes” (just like the Yandex. Search robot spider) millions of pages on the Internet in search of viruses and other malicious programs and compiles the most complete and relevant map of the infected Internet in the world . Even if you read an old letter containing a link that was absolutely safe yesterday, but today leads to an infected page, we will warn you about this and make efforts so that you can avoid infection.

    Also popular now: