February 26, 2015 at 02:28What exactly happens when a user types google.com in the address bar? Part 2
- Transfer
Part 1
The client computer sends a ClientHello message to the server, indicating the TLS version, a list of encryption algorithms, and available compression methods. The server responds with a ServerHello message containing the same data, plus a public server certificate. The certificate contains a public key with which the client will encrypt the remainder of the greeting. The client verifies that the certificate is signed by someone from the list of trusted CAs (Certificate Authority). If the check passes, the client creates a sequence of pseudo-random bytes and encrypts it with the public key of the server. This sequence is used to create a symmetric key.
The server decrypts the sequence with its private key and creates on its basis its copy of the symmetric key. The client sends a Finished message, encrypting the hash of the entire transmission with a symmetric key. The server creates its own hash, and decrypts the hash from the client to verify them. If they are identical, he sends his Finished message, which is also encrypted with a symmetric key.
From this moment on, the TLS session transmits data via the HTTP protocol, encrypted with a symmetric key.
If the user uses a browser from Google, then instead of an HTTP request for the contents of the page, they will be sent a request to upgrade the protocol from HTTP to SPDY. If the client does not support the HTTP protocol, it sends a request to the server in the form:
The remaining headers contain key-value pairs separated by colons. Each pair is separated from the others by a newline. If the client uses older protocols than HTTP / 1.1, then the request will not have the Host header and the version in the first line will be different - 1.0 or 0.9.
In the HTTP / 1.1 protocol, the ability to close the connection after the end of the response is received is specified. The title looks like this:
If the client does not support persistent connections, it must include such a header in any request.
After sending the request and headers, the browser sends an empty line signaling the end of the request content. The server responds with a code indicating the status of the request, followed by response headers:
They are followed by an empty line, after which the HTML content of the www.google.com page is dumped to the client . Then the server closes the connection, or leaves it open, depending on client requests.
If the headers of the HTTP request from the client contained information by which it can be determined that after saving the file in the browser cache it did not change on the server (for example, the browser included the ETag header), the server can respond like this:
and do not include the contents of the file in the response. Then the browser loads the file from the cache.
After parsing the HTML, the browser with the server repeats this procedure for each resource (pictures, files of styles, scripts, etc.), to which there is a link from HTML. Only instead of a GET / HTTP / 1.1 request will it indicate the path to the GET / $ file (URL relative to www.google.com ) HTTP / 1.1.
If a resource is mentioned in HTML from a domain other than www.google.com , the browser repeats the steps from the moment it receives the domain address. The Host header in the request will indicate the desired domain name instead of google.com
HTTPD Server (HTTP Daemon) processes server-side requests. The most popular servers are Apache or nginx for Linux and IIS for Windows. When the server receives the request, it parses it according to the following parameters:
- HTTP request method (GET, POST, HEAD, PUT or DELETE). In our case, it will be a GET
domain, in our case google.com.
- path / page, in our case / (default path)
The server checks:
- the presence of a virtual host corresponding to the requested domain
- this domain accepts GET requests
- the client has the right to make such a request (possible restrictions on ip, authorization, etc. d.)
If the server has a path conversion module (mod_rewrite for Apache or URL Rewrite for IIS), it tries to map the request to one of the specified rules. If a rule is found, the server converts the request according to it.
Then the server reads the file corresponding to the request. If, for example, Google runs PHP, then the server uses PHP to interpret the index file, and returns the result of the interpretation to the client.
After the browser receives all the resources (HTML, CSS, JS, images, etc.), the browser parses text resources (HTML, CSS, JS), and starts the page rendering. To do this, the DOM tree is built, then it is processed, the location of the elements is calculated, and they are displayed on the screen.
The browser function is to provide the selected web resource by requesting it from the server and display it in a window. This is usually an HTML document, but it can be PDF, picture, or other content. The location of the resource is specified by a universal URI.
The way the browser interprets and displays the HTML file is defined in the HTML and CSS specifications, which are supported by the W3C (World Wide Web Consortium).
All browsers usually have:
- the address bar for inserting the URI
- the back and forward buttons
- the ability to bookmark
- the update and stop
buttons - the home button
- user interface - buttons, address bar, menu. This is all but the main part of the window where the page
is visible - the browser engine - handles the interaction of the UI and the render
engine - the render engine - visually displays the requested content
- the network part - HTTP requests, and so on
- the interface backend - draws drop-down menus, windows and other elements. This interface is platform independent, and its operation is provided by the methods of the user interface of a particular OS
- the JavaScript interpreter - parses and executes the code
- data storage - cookies and local storage (localStorage, IndexedDB, WebSQL and FileSystem).
The render engine receives the content of the requested document, usually in 8 Kb pieces. The task of the parser is to turn the markup into a tree. The parser tree is a tree structure of DOM elements and nodes. The DOM, or Document Object Model, is a representation of an HTML document and its elements for other engines, such as JavaScript. Prior to script processing, the DOM has an almost complete one-to-one correspondence with markup.
HTML cannot be parsed by conventional parsers working from top to bottom or bottom to top. The markup is too liberal, browsers make certain markup errors, and the work of scripts sometimes forces the parsing process to return to already passed places. In this regard, the parsing process sometimes changes the input data. The parsing algorithm is described in more detail in the HTML5 specification.
The browser requests external resources (CSS, images, JavaScript, etc.). At this stage, the document becomes interactive and scripts that are executed when the document is ready (deferred mode) begin to be processed. Then the document status is set to complete and the load event is fired.
CSS files, tag content are processed
TLS handshake
The client computer sends a ClientHello message to the server, indicating the TLS version, a list of encryption algorithms, and available compression methods. The server responds with a ServerHello message containing the same data, plus a public server certificate. The certificate contains a public key with which the client will encrypt the remainder of the greeting. The client verifies that the certificate is signed by someone from the list of trusted CAs (Certificate Authority). If the check passes, the client creates a sequence of pseudo-random bytes and encrypts it with the public key of the server. This sequence is used to create a symmetric key.
The server decrypts the sequence with its private key and creates on its basis its copy of the symmetric key. The client sends a Finished message, encrypting the hash of the entire transmission with a symmetric key. The server creates its own hash, and decrypts the hash from the client to verify them. If they are identical, he sends his Finished message, which is also encrypted with a symmetric key.
From this moment on, the TLS session transmits data via the HTTP protocol, encrypted with a symmetric key.
HTTP protocol
If the user uses a browser from Google, then instead of an HTTP request for the contents of the page, they will be sent a request to upgrade the protocol from HTTP to SPDY. If the client does not support the HTTP protocol, it sends a request to the server in the form:
GET / HTTP/1.1
Host: google.com
Connection: close
[остальные заголовки]
The remaining headers contain key-value pairs separated by colons. Each pair is separated from the others by a newline. If the client uses older protocols than HTTP / 1.1, then the request will not have the Host header and the version in the first line will be different - 1.0 or 0.9.
In the HTTP / 1.1 protocol, the ability to close the connection after the end of the response is received is specified. The title looks like this:
Connection: close
If the client does not support persistent connections, it must include such a header in any request.
After sending the request and headers, the browser sends an empty line signaling the end of the request content. The server responds with a code indicating the status of the request, followed by response headers:
200 OK
[заголовки ответа]
They are followed by an empty line, after which the HTML content of the www.google.com page is dumped to the client . Then the server closes the connection, or leaves it open, depending on client requests.
If the headers of the HTTP request from the client contained information by which it can be determined that after saving the file in the browser cache it did not change on the server (for example, the browser included the ETag header), the server can respond like this:
304 Not Modified
[заголовки ответа]
and do not include the contents of the file in the response. Then the browser loads the file from the cache.
After parsing the HTML, the browser with the server repeats this procedure for each resource (pictures, files of styles, scripts, etc.), to which there is a link from HTML. Only instead of a GET / HTTP / 1.1 request will it indicate the path to the GET / $ file (URL relative to www.google.com ) HTTP / 1.1.
If a resource is mentioned in HTML from a domain other than www.google.com , the browser repeats the steps from the moment it receives the domain address. The Host header in the request will indicate the desired domain name instead of google.com
HTTP Server Request Handle
HTTPD Server (HTTP Daemon) processes server-side requests. The most popular servers are Apache or nginx for Linux and IIS for Windows. When the server receives the request, it parses it according to the following parameters:
- HTTP request method (GET, POST, HEAD, PUT or DELETE). In our case, it will be a GET
domain, in our case google.com.
- path / page, in our case / (default path)
The server checks:
- the presence of a virtual host corresponding to the requested domain
- this domain accepts GET requests
- the client has the right to make such a request (possible restrictions on ip, authorization, etc. d.)
If the server has a path conversion module (mod_rewrite for Apache or URL Rewrite for IIS), it tries to map the request to one of the specified rules. If a rule is found, the server converts the request according to it.
Then the server reads the file corresponding to the request. If, for example, Google runs PHP, then the server uses PHP to interpret the index file, and returns the result of the interpretation to the client.
Behind the scenes of the browser
After the browser receives all the resources (HTML, CSS, JS, images, etc.), the browser parses text resources (HTML, CSS, JS), and starts the page rendering. To do this, the DOM tree is built, then it is processed, the location of the elements is calculated, and they are displayed on the screen.
Browser
The browser function is to provide the selected web resource by requesting it from the server and display it in a window. This is usually an HTML document, but it can be PDF, picture, or other content. The location of the resource is specified by a universal URI.
The way the browser interprets and displays the HTML file is defined in the HTML and CSS specifications, which are supported by the W3C (World Wide Web Consortium).
All browsers usually have:
- the address bar for inserting the URI
- the back and forward buttons
- the ability to bookmark
- the update and stop
buttons - the home button
High level browser structure
- user interface - buttons, address bar, menu. This is all but the main part of the window where the page
is visible - the browser engine - handles the interaction of the UI and the render
engine - the render engine - visually displays the requested content
- the network part - HTTP requests, and so on
- the interface backend - draws drop-down menus, windows and other elements. This interface is platform independent, and its operation is provided by the methods of the user interface of a particular OS
- the JavaScript interpreter - parses and executes the code
- data storage - cookies and local storage (localStorage, IndexedDB, WebSQL and FileSystem).
HTML parsing (parser)
The render engine receives the content of the requested document, usually in 8 Kb pieces. The task of the parser is to turn the markup into a tree. The parser tree is a tree structure of DOM elements and nodes. The DOM, or Document Object Model, is a representation of an HTML document and its elements for other engines, such as JavaScript. Prior to script processing, the DOM has an almost complete one-to-one correspondence with markup.
Parser algorithm
HTML cannot be parsed by conventional parsers working from top to bottom or bottom to top. The markup is too liberal, browsers make certain markup errors, and the work of scripts sometimes forces the parsing process to return to already passed places. In this regard, the parsing process sometimes changes the input data. The parsing algorithm is described in more detail in the HTML5 specification.
At the end of parsing
The browser requests external resources (CSS, images, JavaScript, etc.). At this stage, the document becomes interactive and scripts that are executed when the document is ready (deferred mode) begin to be processed. Then the document status is set to complete and the load event is fired.
CSS interpretation
CSS files, tag content are processed