What I understood and what problems I encountered creating a Hacker News clone



    From the translator: this article is an abridged translation of the original post by web developer Jesse Horn. His work and at the same time his hobby is web design. Jesse often shares her experience and lessons learned with other programmers, both experienced and beginners.

    Some time ago I wrote a rather lengthy post on Hacker News, which received solid support from readers. I drew attention to the possibilities of this resource and decided to try to create its clone in order to get new experience and knowledge.

    Skillbox recommends: Practical annual course PHP developer .
    We remind: for all readers of "Habr" - a discount of 10,000 rubles when writing to any Skillbox course on the promotional code "Habr".

    Initially, it seemed to me an excellent goal, where the toolkit and tactics of work were predetermined. I previously worked on smaller tasks using different languages ​​and frameworks. The only thing that didn’t exist was a major project that would allow to bring it all together.

    I decided to work on a new project from scratch using the Crystal language. This is a handy tool, which is also a new one. He is fast, the principles of working with him are similar to Ruby. It is statistically typed plus is open source. But just a new language is also not enough, so I decided to complicate my task, and do not a clone of Hacker News, but an improved version of this resource.

    By the way, I thought that creating an ordinary clone would be an easy task. And the fact that this is not so, I realized a little later, starting work.

    Do not throw too high

    I do not want to say that you should not set ambitious goals. Try to reach the stars, through or without thorns. Dreaming is useful, trying to achieve what I could not do earlier is necessary for successful advancement on the professional ladder.

    I said that you should not try to immediately rush to the embrasure of performing a complex and large task. Choose what you need at a particular moment to implement the idea. Further, I recommend dividing significant targets into smaller ones. If you have too much code in which a large amount of "garbage", then try to simplify and structure everything.

    I cannot even put into words how important it is to divide a large task into several smaller ones.

    In my case, I ran into various new problems that I had not met before. First, I worked Crystal only once - when I created an application, the complexity of which slightly exceeded the complexity of a program like “Hello, world!”. In the past I used Python, Lua, PHP. The new language was the first obstacle to the project.

    The second was the framework. Crystal, being a new language, still does not have too many frameworks and documentation for them. I decided to use Kemal for my clone. I hoped that the framework is intuitive, because in my work I met just such. But here was another case - it was not easy to understand certain things. For example, some parts of the code looked quite normal, but for some reason did not work. And finding the cause of the problem on the Internet was difficult for the same reason - the novelty of the language and the framework itself. In some cases, I had to stop for a long time and deal with the cause of the problem. Until now, I have never looked at the sources, for example, the same Flask.

    Third obstacle- specific goals that I have chosen for the project. In order to make my clone more useful, I decided to add a number of functional features. I worked with Hacker News before, so I know a lot about the advantages and disadvantages of this resource. It occurred to me that it would be good to use the analytical data provided by GitHub. Using them, I could add statistics on the number of views and user activity on the site.

    So I decided that my clone should show at least views and clicks for each post. Such a function would probably be useful for users, and its implementation would be good practice for me. But I did not think it would take so much time! Now the project is a proof-of-concept rather than a ready-made resource. In addition to views and clicks, I also decided to add a display of comments that appear so that they appear in real-time. If it were not for this additional task, the project would have been completed much faster.

    Now the project supports the authorization, placement and sorting of posts, interaction with other users by commenting and viewing profiles, etc.

    Fourth problem, which I encountered - chaos. Migrations, models, views, API controllers, CSS, and JavaScript code accumulate very quickly. And it all turns to mush. I tried to solve the problem, and I almost got it. But part of the problem areas in the code of my project still occur - you can notice it without any problems. So, in many places the code is repeated, endpoints are not organized, naming is complicated, which leads to an increase in the amount of code and a large number of queries to the database. Gradually, I solve this problem.

    In many places the code repeats.

    When the project started, I solved problems as they appeared. For example, it was necessary to check whether the user is authenticated - and I found the original solution, which took three lines or so depending on the route. I was in a hurry to implement the solutions found.



    In general, I had to follow the principles of DRY from the very beginning. Do not repeat! I needed to write an intermediate program that would do what was necessary, and then simply applied the solution found when it was required. In the future, I will adjust my project based on this principle.

    Endpoint naming and organization

    I used to develop an API on the Pioneer team. We used REST- architecture style, which I actually swore, being a new member of the Pioneer team. In this team, I worked on small elements of a huge application. But in my new project I had to write everything from scratch.

    A “RESTful API” must have a Uniform Interface, as stated in the Wiki. In my project, I organized everything in such a way that endpoints were defined in source files, which were called according to function or purpose. Registration and exit from the application were registered in "auth.cr". User login was registered in "user.cr".

    Undoubtedly, this part requires processing. In the future, I will arrange everything so that the actions relate to the endpoint "/ user", simply because it all has to do with User. For example, to improve the structure of the project I will divide the code into more convenient and manageable parts with a large number of files. I will create the “src / user / auth /” folder, which will contain the login, registration and logout functions of the user from the application. Too many requests

    1_6dy_OCE7_Cjj_Df7_A0s_Aj_3i_Q



    In this project, I'm trying to scale my ideas. There are currently too many requests for the database. So, if the application had a hundred users, I would just go crazy trying to find out how many database queries are being executed every few seconds. Now the client updates the status of messages every 10 minutes. In addition, the post cyclically determines whether it was viewed by the user. During each check, a request to the database occurs; this is performed in accordance with the current filter. And all this significantly loads the server.



    All that is described above are just examples of the problems that I face all the time. I can not describe them in full. Nevertheless, I am now pondering over documenting my problems and solutions. I hope that all this will help other programmers to see and understand my mistakes and avoid their own. I know that if I use a combination of Redis and MySQL or Postgres, I can speed up the execution of queries. But for now, I’ll stop here.

    I hope that you liked the article and you were able to learn something useful for yourself.

    Skillbox recommends:

    Tags:

    Also popular now: