May 1, 2019 at 21:59How web applications work
This article is for beginners and those who want a little guidance in the terms and technologies of modern web applications. The article describes how web applications differ from sites, what web applications are, what they consist of and how they work.
For me, a site is primarily something informative and static: a business card of a company, a site for recipes, a city portal or a wiki. A set of pre-prepared HTML files that are on a remote server and are given to the browser upon request.
Sites contain various statics, which like an HTML file are not generated on the fly. Most often these are pictures, CSS-files, JS-scripts, but there can be any other files: mp3, mov, csv, pdf.
Blogs, business cards with a contact form, landing pages with a bunch of effects, I also refer to sites for simplicity. Although, unlike completely static sites, they already include some kind of business logic.
And a web application is something technically more complex. Here HTML pages are generated on the fly depending on the user's request. Email clients, social networks, search engines, online stores, online business programs, these are all web applications.
Web applications can be divided into several types, depending on different combinations of its main components:
In development, a framework is a set of ready-made libraries and tools that help you create web applications. For example, I will describe the principle of operation of the Django framework written in the Python programming language.
The first step is a request from the user to the router (URL dispatcher), which decides which function to process the request to call. The decision is made on the basis of a list of rules consisting of a regular expression and the name of the function: if such url, then such a function.
The function that is called by the router is called view. Any business logic can be contained inside, but most often it is one of two things: either data is taken from the database, prepared and returned to the front; or a request came with data from some form, this data is checked and stored in the database.
Application data is stored in a database (DB). Most commonly used relational databases. This is when there are tables with predefined columns and these tables are interconnected through one of the columns.
Data in the database can be created, read, modified and deleted. Sometimes, to denote these actions, one may come across the abbreviation CRUD (Create Read Update Delete). To query data in the database, a special SQL language (structured query language) is used.
In Django, models are used to work with databases. They allow you to describe tables and make queries on the usual python developer, which is much more convenient. You have to pay for this convenience: such queries are slower and limited in capabilities compared to using pure SQL.
The data received from the database is prepared in view for sending to the front. They can be substituted into a template and sent as an HTML file. But in the case of a one-page application, this happens only once, when an HTML page is generated, to which all JS scripts are connected. In other cases, the data is serialized and sent in JSON format.
The client part of the application is scripts written in the Javascript (JS) programming language and executed in the user's browser. Previously, all client logic was based on the use of the JQuery library, which allows you to work with the DOM, animation on the page and make AJAX requests.
DOM (document object model) is the structure of an HTML page. Working with the DOM means finding, adding, modifying, moving, and deleting HTML tags.
AJAX (asynchronous javascript and XML) is a common name for technologies that allow you to make asynchronous (without reloading the page) requests to the server and exchange data. Since the client and server parts of the web application are written in different programming languages, for the exchange of information, it is necessary to convert the data structures (for example, lists and dictionaries) in which it is stored into JSON format.
JSON (JavaScript Object Notation) is a universal format for exchanging data between a client and a server. It is a simple string that can be used in any programming language.
Serialization is the conversion of a list or dictionary into a JSON string. For example:
Dictionary:
Serialized string:
Deserialization is the inverse transformation of a string into a list or dictionary.
By manipulating the DOM, you can completely control the content of the pages. Using AJAX, you can exchange data between the client and server. With these technologies you can already create a SPA. But when creating a complex application, jQuery-based front-end code quickly becomes confusing and difficult to maintain.
Fortunately, Javascript frameworks have replaced JQuery: Backbone Marionette, Angular, React, Vue, and others. They have a different philosophy and syntax, but they all allow you to manage data on the front end with much greater convenience, they have template engines and tools for creating navigation between pages.
An HTML template is a smart HTML page in which variables are used instead of specific values and various operators are available: if , the for loop, and others. The process of getting an HTML page from a template when variables are substituted and operators are applied is called template rendering.
The resulting page is shown to the user. Switching to another section in SPA is using a different template. If you need to use other data in the template, then they are requested from the server. All form submissions with data are AJAX requests to the server.
The client communicates with the server via HTTP. The basis of this protocol is the request from the client to the server and the response of the server to the client.
For queries, they usually use GET methods if we want to get data, and POST if we want to change the data. The request also indicates Host (site domain), request body (if it is a POST request) and a lot of additional technical information.
Modern web applications use the HTTPS protocol, an extended version of HTTP with support for SSL / TLS encryption. Using an encrypted data transmission channel, regardless of the importance of this data, has become a good form on the Internet.
There is another request that is made before HTTP. This is a DNS (domain name system) query. It is needed to obtain the ip address to which the requested domain is bound. This information is stored in the browser and we no longer spend time on this.
When a request from the browser reaches the server, it does not immediately get to Django. First, it is processed by the Nginx web server. If a static file is requested (for example, a picture), then Nginx itself sends it in response to the client. If the request is not static, then Nginx must proxy (transfer) it to Django.
Unfortunately, he does not know how. Therefore, another layer program is used - the application server. For example, for python applications, it could be uWSGI or Gunicorn. And now they send the request to Django.
After Django processed the request, he returns a response with an HTML page or data, and a response code. If all is well, then the response code is 200; if the page is not found, then - 404; if an error occurs and the server was unable to process the request, then - 500. These are the most common codes.
Another technology that we constantly encounter, which is present both in web applications and software, and at the processor level in our computers and smartphones.
Cache is a concept in development when frequently used data, instead of getting it out of the database each time, calculated or prepared in another way, is stored in a quickly accessible place. Some examples of cache usage:
1. How web applications differ from sites
For me, a site is primarily something informative and static: a business card of a company, a site for recipes, a city portal or a wiki. A set of pre-prepared HTML files that are on a remote server and are given to the browser upon request.
Sites contain various statics, which like an HTML file are not generated on the fly. Most often these are pictures, CSS-files, JS-scripts, but there can be any other files: mp3, mov, csv, pdf.
Blogs, business cards with a contact form, landing pages with a bunch of effects, I also refer to sites for simplicity. Although, unlike completely static sites, they already include some kind of business logic.
And a web application is something technically more complex. Here HTML pages are generated on the fly depending on the user's request. Email clients, social networks, search engines, online stores, online business programs, these are all web applications.
2. What are web applications
Web applications can be divided into several types, depending on different combinations of its main components:
- Backend (backend or server side of the application) runs on a remote computer, which can be located anywhere. It can be written in different programming languages: PHP, Python, Ruby, C # and others. If you create an application using only the server side, then as a result of any transitions between sections, form submissions, data updates, the server will generate a new HTML file and the page in the browser will reload.
- Frontend (frontend or client side of the application) runs in the user's browser. This part is written in the Javascript programming language. An application can consist only of a client part if it is not required to store user data for longer than one session. It can be, for example, photo editors or simple toys.
- Single page application (SPA or single page application). A more interesting option is when both the backend and frontend are used. Using their interaction, you can create an application that will work without any page reloads in the browser. Or in a simplified version, when transitions between partitions cause reboots, but any actions in the partition do without them.
3. Pyhon framework Django aka backend
In development, a framework is a set of ready-made libraries and tools that help you create web applications. For example, I will describe the principle of operation of the Django framework written in the Python programming language.
The first step is a request from the user to the router (URL dispatcher), which decides which function to process the request to call. The decision is made on the basis of a list of rules consisting of a regular expression and the name of the function: if such url, then such a function.
The function that is called by the router is called view. Any business logic can be contained inside, but most often it is one of two things: either data is taken from the database, prepared and returned to the front; or a request came with data from some form, this data is checked and stored in the database.
Application data is stored in a database (DB). Most commonly used relational databases. This is when there are tables with predefined columns and these tables are interconnected through one of the columns.
Data in the database can be created, read, modified and deleted. Sometimes, to denote these actions, one may come across the abbreviation CRUD (Create Read Update Delete). To query data in the database, a special SQL language (structured query language) is used.
In Django, models are used to work with databases. They allow you to describe tables and make queries on the usual python developer, which is much more convenient. You have to pay for this convenience: such queries are slower and limited in capabilities compared to using pure SQL.
The data received from the database is prepared in view for sending to the front. They can be substituted into a template and sent as an HTML file. But in the case of a one-page application, this happens only once, when an HTML page is generated, to which all JS scripts are connected. In other cases, the data is serialized and sent in JSON format.
4. Javascript frameworks aka frontend
The client part of the application is scripts written in the Javascript (JS) programming language and executed in the user's browser. Previously, all client logic was based on the use of the JQuery library, which allows you to work with the DOM, animation on the page and make AJAX requests.
DOM (document object model) is the structure of an HTML page. Working with the DOM means finding, adding, modifying, moving, and deleting HTML tags.
AJAX (asynchronous javascript and XML) is a common name for technologies that allow you to make asynchronous (without reloading the page) requests to the server and exchange data. Since the client and server parts of the web application are written in different programming languages, for the exchange of information, it is necessary to convert the data structures (for example, lists and dictionaries) in which it is stored into JSON format.
JSON (JavaScript Object Notation) is a universal format for exchanging data between a client and a server. It is a simple string that can be used in any programming language.
Serialization is the conversion of a list or dictionary into a JSON string. For example:
Dictionary:
{
'id': 1,
'email': 'mail@example.com'
}Serialized string:
'{"id": 1, "email": "mail@example.com"}'Deserialization is the inverse transformation of a string into a list or dictionary.
By manipulating the DOM, you can completely control the content of the pages. Using AJAX, you can exchange data between the client and server. With these technologies you can already create a SPA. But when creating a complex application, jQuery-based front-end code quickly becomes confusing and difficult to maintain.
Fortunately, Javascript frameworks have replaced JQuery: Backbone Marionette, Angular, React, Vue, and others. They have a different philosophy and syntax, but they all allow you to manage data on the front end with much greater convenience, they have template engines and tools for creating navigation between pages.
An HTML template is a smart HTML page in which variables are used instead of specific values and various operators are available: if , the for loop, and others. The process of getting an HTML page from a template when variables are substituted and operators are applied is called template rendering.
The resulting page is shown to the user. Switching to another section in SPA is using a different template. If you need to use other data in the template, then they are requested from the server. All form submissions with data are AJAX requests to the server.
5. How the client and server communicate with each other
The client communicates with the server via HTTP. The basis of this protocol is the request from the client to the server and the response of the server to the client.
For queries, they usually use GET methods if we want to get data, and POST if we want to change the data. The request also indicates Host (site domain), request body (if it is a POST request) and a lot of additional technical information.
Modern web applications use the HTTPS protocol, an extended version of HTTP with support for SSL / TLS encryption. Using an encrypted data transmission channel, regardless of the importance of this data, has become a good form on the Internet.
There is another request that is made before HTTP. This is a DNS (domain name system) query. It is needed to obtain the ip address to which the requested domain is bound. This information is stored in the browser and we no longer spend time on this.
When a request from the browser reaches the server, it does not immediately get to Django. First, it is processed by the Nginx web server. If a static file is requested (for example, a picture), then Nginx itself sends it in response to the client. If the request is not static, then Nginx must proxy (transfer) it to Django.
Unfortunately, he does not know how. Therefore, another layer program is used - the application server. For example, for python applications, it could be uWSGI or Gunicorn. And now they send the request to Django.
After Django processed the request, he returns a response with an HTML page or data, and a response code. If all is well, then the response code is 200; if the page is not found, then - 404; if an error occurs and the server was unable to process the request, then - 500. These are the most common codes.
6. Caching in web applications
Another technology that we constantly encounter, which is present both in web applications and software, and at the processor level in our computers and smartphones.
Cache is a concept in development when frequently used data, instead of getting it out of the database each time, calculated or prepared in another way, is stored in a quickly accessible place. Some examples of cache usage:
- In Django, a request came to receive data for the chart in the report. We get the data from the database, prepare it and put it into the database with quick access, for example, memcached for 1 hour. At the next request, we will immediately get them from memcached and send them to the frontend. If we find out that the data has ceased to be relevant, we invalidate it (delete it from the cache).
- For caching static files, CDN (content delivery network) providers are used. These are servers located around the world and optimized for the distribution of statics. Sometimes it’s more efficient to put pictures, videos, JS-scripts on CDN instead of your server.
- All browsers enable caching of static files by default. Thanks to this, opening the site is not the first time, everything loads noticeably faster. The minus for the developer is that with the cache turned on, changes made in the code are not always immediately visible.