Build Go web app in 2017
A couple of weeks ago, I started developing another web application, only already on Go . Being mainly a backend developer, I did not often write entire web applications, so each such case was more like a challenge. At such moments, I would like someone to write a guide on web development for people who do not have the opportunity to delve into the intricacies of great design all day, but who just need to create a working, functional website without too much fuss.
I decided to take this opportunity and make such a guide in which to start from scratch a web application in the way it should be done in 2017 (in my understanding). I spent many hours digging through the things that I had previously avoided, just so that at least once in many years I could confidently say that I have my own opinion and experience in this matter and get a working recipe for myself , which may be useful not only to me.
The article begins a short series covering what I learned in the process. This first post is a general introduction describing the current state of affairs, problems, and why I consider Go a good choice. Subsequent articles will be more detailed and contain more code. I am curious how much my experience correlates with yours; maybe I'm wrong about something, so feel free to comment.
If you are only interested in the code, it is here .
Python and Ruby
A year ago, I would recommend Python or Ruby as a web application environment. There may be other similar languages, but from my point of view, Python and Ruby dominate the world.
Most of the time, the main task of a web application was to build web pages using server-side HTML layout. Both Python and Ruby are very well suited for extracting data from a database and turning it into a bunch of HTML code using templates. There are many frameworks / tools to choose from, e.g. Rails, Django, Sinatra, Flask, etc. etc.
Although these languages have certain significant limitations, such as GIL , the ease with which they solve the problem of generating HTML is much more valuable than the compromises that you have to make.
GIL (Global Interpreter Lock) tears a separate mention. Of course, this is the biggest limitation of any solution in Python or Ruby, but this is a very slippery topic, people often prefer to pretend that there is no problem. And if we are talking about this, emotions usually hit over the edge, in the Ruby and Python communities there are endless discussions on the topic of GIL.
For those unfamiliar with this problem - GIL allows you to do just one thing at a time. When you create threads and they “look” as parallel executing, in fact the interpreter still executes the instructions sequentially. This means that one process can use only one CPU.
Alternative implementations exist, such as those based on the JVM, but they are not often used. I don’t know exactly why, maybe they are not fully compatible or probably do not support C extensions correctly, and they still may have GIL. I'm not sure, but as far as I can tell, the implementation in C is usually used all the same. To make an interpreter without GIL, you have to completely rewrite it, and this can already change the behavior of the language (in my naive understanding), and therefore it seems to me that GIL will stay.
Web applications of any significant scale necessarily require the ability to service requests in parallel, using the capabilities of each available CPU on the machine. So far, the only possible solution is to run several instances of the application as separate processes.
This is usually done using additional software such as Unicorn / Gunicorn, with each process listening on its own port and starting behind some kind of connection balancer, such as Nginx and / or Haproxy. Alternatively, this can be done through Apache and its modules (such as mod_python or mod_wsgi), which is difficult anyway. Such applications typically rely on the database server as an arbiter for any competitive-sensitive tasks. When implementing caching, in order not to store multiple copies of the same on the same server, a shared-memory store is required, such as Memcached or Redis, and usually both. Also, such applications can not do background processing, for this there is a separate set of tools, such as Resque. And then all these components require monitoring to make sure that it all works. Logs must be consolidated, and they have their own additional tools. Given the inevitable complexity of this setup, a configuration manager such as Chef or Puppet is also required. And yet, these kits are generally not able to support a large number of long-term connections - a problem known asC10K .
As a result, a simple web-based database application requires a whole bunch of components before it can serve the Hello World! Page. And almost all of this is due to the GIL.
The advent of single-page applications
A look at Golang
Go is gradually eroding the established world of web applications. It natively supports parallel execution, eliminating the need for almost all of the components commonly used to work with GIL constraints.
Programs on Go are binaries that are natively launched, so you do not need to install anything language-specific on the server. The problem of ensuring the correct version of the runtime required by the application disappears; There is no separate runtime environment - it is built into the binary. Go programs can easily and elegantly run tasks in the background, so there is no need for tools like Resque. These programs run as a single process, so caching becomes trivial, which means Memcached or Redis are not needed. Go can manage an unlimited number of concurrent connections, eliminating the need for front-end protection such as Nginx.
With Go, a tall multi-layered tower of Python, Ruby, Bundler, Virtualenv, Unicorn, WSGI, Resque, Memcached, Redis, etc., etc. reduced to just one binary. The only third-party component that is usually still needed is the database (I would recommend PostgreSQL). It is important to note that all these tools can still be used, but you can do without Go without them.
The launch time of such a Go-program will most likely be an order of magnitude greater than any Python / Ruby application, it will require less memory and lines of code.
Ok, but is there a popular framework?
The short answer is this: the framework is optional and not recommended. There are many projects that claim to be a great framework, but I think it's better to do without them. This is not only my personal opinion, I find this opinion quite common in the Go community.
You need to understand why frameworks were created at all. In the Python / Ruby world, this happened because these languages were not originally designed to serve web pages, and many external components were needed to solve this problem. The same can be said about Java, which, like Python and Ruby, is as old as the web, as we know it, or even a little older.
As far as I remember, early versions of Python out of the box did not provide anything for working with a database, there were no templates, HTTP support was confusing, working with a network was non-trivial, even encryption was illegal then, and in general, a lot was missing. The framework provided all these necessary pieces and set language-specific development rules for all common web application variations.
Go, on the other hand, was created by people who already had experience and were versed in web development. It includes almost everything you need. One or two external packages may be needed to solve some specific tasks, such as OAuth, but in no case this pair of packages is not a "framework".
If all of the above regarding frameworks does not sound convincing enough, it is useful to consider the learning curve for the frameworks and the risks. It took me about two years to build a relationship with Rails. Frameworks can become abandoned and outdated, and porting an application to a new framework is difficult, and sometimes impossible. Given how fast things are changing in information technology, the framework certainly should not be chosen lightly.
Undoubtedly, frameworks make some things easier, especially in the typical world of CRUD applications for business, where applications have many pages with many fields, manipulate data in complex and constantly changing database schemes. Not sure if Go is a good choice in such an environment, especially if performance and scalability are not a priority.
What about database and ORM?
Similar to frameworks, ORMs in Go are not very common. For starters, Go does not support objects - what is indicated by O in the abbreviation ORM.
I know that instead of using the convenient
User.find(:all).filter..., which is provided by something like ActiveRecord, writing SQL manually is something unheard of in some communities, but I still think that this attitude should change. SQL is a great language. Dealing with SQL directly is not so difficult, and in return we get more freedom and opportunities. Perhaps the most tedious part of such direct work is copying data from the database cursor into structures, but the sqlx project is very useful here .
In my opinion, the article describes in sufficient detail the current situation on the server side. I think it’s better to separate the client part into a separate post, therefore for today - that’s all. To summarize, we build the application with approximately the following requirements:
- Minimal dependency on third-party packages.
- Without a web framework.
- PostgreSQL as a database.
- Single page application.