We read Habr in the mail

    image
    All read Habr. And including me. But once the proxy was closed access to it. In this regard, it was decided to do such a thing that would make it possible to read Habr's articles.
    The result was a service running on a home computer that checks new articles on Habré, sends the article headers to e-mail, and also sends the articles themselves upon request. Communication with the service only by mail. More details under the cut.

    The service is running on the home computer. The service starts four tasks that run after a specified period of time:
    1. (Every 15 minutes) Checking for new articles on hubs. Numbers and titles of new articles are stored in the database
    2. (Every 5 minutes) Sending to subscribers numbers and names of new articles that have not yet been sent
    3. (Every 2 minutes) Receive request from test mail. The request is saved in the database
    4. (Every 2 minutes) Processing the request and sending the full article to the sender's mail

    More about tasks:
    1. There are a lot of hubs, it’s been very long to download articles on all A small list was selected for testing. Only 47 hubs. When completing the first task, the service downloads the article pages of the hubs, parses the text and puts new articles in the database (not the article itself, but only the title and number)
    2. When completing the second task, the service selects the numbers and headings of new articles (which have not yet been sent) from the database and sends them to all subscribers
    3. During the third task, new letters are downloaded from the special mail habrpost@mail.ru. The subject of the letter and the sender are saved in the database
    4. When completing the fourth task, an analysis of the received letters takes place. Depending on the subject of the letter, the task may result in the following:
      • The sender subscribes to the newsletter to receive headlines for new articles
      • Sender unsubscribes
      • The sender receives an article in the mail. If the article has already been downloaded earlier, then the already downloaded article will be sent. If the article has not yet been downloaded, then the HTML + article with all the images and CSS styles will be downloaded. All this is packaged in an archive
      • The sender receives a newly downloaded article in the mail (the article will be fully downloaded again, even if it was previously downloaded)


    Well, the most interesting. I will be glad to feedback. You can test the service. Here is a list of commands:

    Description:
    - Write a new letter to habrpost@mail.ru;
    - The team is written in the subject line of the letter;
    - The team is recorded only in numbers;

    Description of the teams:
    1 - subscribe to the newsletter (receive the names of new articles by mail);
    0 - unsubscribe from the newsletter;
    123456 - receive article No. 123456 by mail;
    123456 1 - re-download article No. 123456 and receive the article in the mail;

    PS On October 10, he stopped the service in order to analyze the work of the service. The results can be found in another article .

    Also popular now: