Servo Browser Engine Architecture
From the translator . Let me introduce to the court of the habrasociety the translation of part of the documentation on the Servo browser engine. This engine is developed by the Mozilla community in the Rust language, and is perhaps the largest active project in this language. This document talks about the architecture of the engine, how developers use Rust in conjunction with C ++, and what difficulties they encountered during development. The original is available on the github project wiki .
This is a very superficial overview of the Servo architecture. Servo remains the prototype, and some parts of the architecture are not yet represented as code. Some important aspects of the system have not yet been discussed in detail.
Servo is a research project to develop a new browser engine. Our goal is to create an architecture that benefits from concurrency and at the same time eliminates common sources of bugs and vulnerabilities associated with incorrect memory management and race conditions.
Since C ++ is not well suited to prevent these problems, Servo is written in Rust , a new language designed specifically for Servo. Rust provides a task-parallel infrastructure and a powerful type system that provides memory security and freedom from race conditions.
When making decisions during the design process, we give priority to the features of the modern web platform, which boil down to high-performance, dynamic, rich multimedia applications, possibly to the detriment of what cannot be optimized. We want to know what a fast and responsive web platform is and implement it.
Servo clearly does not pretend to create a full-fledged browser (with the exception of the needs of the demonstration or experiments). On the contrary, it aims to create a holistic, embeddable engine. Although Servo is a research project, it is designed to be usable — the code we write must be of high enough quality to ultimately reach users.
Competitiveness is the division of tasks into parts for alternating execution. Concurrency is the simultaneous execution of several parts of work to increase speed. Some ideas in this direction that we are studying or plan to consider:
Each instance of constellation can be considered as a separate tab or window. It manages a task pipeline that accepts input, runs JavaScript for the DOM, arranges elements, builds display lists, renders display lists in tiles, and displays the final image to the surface.
The conveyor consists of four main tasks:
In this pipeline, two complex data structures are used in the interaction of tasks: the DOM and the display list. The DOM is passed from the content processing task to the placement processing task, and the display list from the placement task to the renderer. The search for an effective and type-safe way to represent, share and / or transfer these two structures is one of the main difficulties of this project.
Servo's DOM is a tree with support for node versioning, which can be shared between one writer and several readers. The DOM uses a copy-on-write strategy to allow the writer to modify the DOM in parallel with readers. The content processing task always writes, and the placement tasks or their subtasks are always read.
DOM nodes are Rust values whose lifetime is controlled by the JavaScript garbage collector. JavaScript accesses the DOM nodes directly - there is no XPCOM or similar infrastructure.
The DOM interface is currently not type-safe - it is possible to incorrectly manipulate nodes, which will lead to dereferencing of invalid pointers. The elimination of this insecurity is a high priority and necessary goal of the project; since DOM nodes have a complex life cycle this will lead to some difficulties.
Servo rendering is fully controlled by the display list, a sequence of high-level commands created by the layout task. Servo's display list is immutable, so it can be shared between competing renderers and it contains everything you need to display. This differs from the WebKit renderer, which does not use the display list and the Gecko renderer, which uses the display list, but also, during rendering, accesses additional information, for example, directly to the DOM.
We currently use SpiderMonkey, although plug-in engines are a long-term low-priority task. Each content processing task gets its own JavaScript runtime. DOM bindings use the native engine API instead of XPCOM. Automatic generation of bindings from WebIDL in priority.
Just like Chromium and WebKit2, we intend to have a trusted process application and several less trusted process engines. The high-level API will, in fact, be based on IPC, and most likely with non-IPC implementations for testing and the single-process version, although it is assumed that the most serious users will use several processes. The engine process will use sandbox mechanisms provided by the operating system to limit access to system resources.
At the moment, we don’t intend to go into extremes with regard to the sandbox, like the Chromium developers, mainly because tying it to the sandbox requires a lot of work of developers (in particular on low-priority platforms like Windows XP or old Linux) and other aspects of the project are more priority . The Rust type system also adds an important layer of defense against memory protection vulnerabilities. This alone does not make the sandbox any less important in terms of protection against unsafe code, bugs in the type system and third-party libraries and libraries on the local computer, but this significantly reduces the possibility of attacks on Servo compared to other browser engines. In addition, we are concerned about performance related to some sandboxing techniques (e.g.
Web pages depend on many external resources, with a large number of mechanisms for receiving and decoding. These resources are cached at several levels - on disk, in memory, and / or in decoded form. Within a parallel browser, these resources should be distributed among competitive tasks.
Traditionally, browsers have been single-threaded, performing I / O on the “main thread”, where most of the calculations are also performed. This leads to problems with delays. Servo does not have a “main stream” and all external resources are loaded in a single resource manager .
Browsers use a lot of caches, and the task-based Servo architecture implies that it will probably use more caches than existing engines (we may have both a global cache based on a separate task and a local cache for the task , which stores the results from the global cache, to avoid calls through the scheduler). Servo should have a unified caching mechanism, with custom caches that will work in environments with a small amount of memory.
This is a very superficial overview of the Servo architecture. Servo remains the prototype, and some parts of the architecture are not yet represented as code. Some important aspects of the system have not yet been discussed in detail.
Overview and Goals
Servo is a research project to develop a new browser engine. Our goal is to create an architecture that benefits from concurrency and at the same time eliminates common sources of bugs and vulnerabilities associated with incorrect memory management and race conditions.
Since C ++ is not well suited to prevent these problems, Servo is written in Rust , a new language designed specifically for Servo. Rust provides a task-parallel infrastructure and a powerful type system that provides memory security and freedom from race conditions.
When making decisions during the design process, we give priority to the features of the modern web platform, which boil down to high-performance, dynamic, rich multimedia applications, possibly to the detriment of what cannot be optimized. We want to know what a fast and responsive web platform is and implement it.
Servo clearly does not pretend to create a full-fledged browser (with the exception of the needs of the demonstration or experiments). On the contrary, it aims to create a holistic, embeddable engine. Although Servo is a research project, it is designed to be usable — the code we write must be of high enough quality to ultimately reach users.
Concurrency and Competition Strategies
Competitiveness is the division of tasks into parts for alternating execution. Concurrency is the simultaneous execution of several parts of work to increase speed. Some ideas in this direction that we are studying or plan to consider:
- Task based architecture . The main components of the system should be
represented in the form of actors, with isolated memory, with clear boundaries for the possibility of failure and recovery. It will also contribute to weak binding of system components, allowing us to replace them for the purpose of experimentation and research. It is implemented. - Competitive rendering . Rendering and compositing are performed in different streams, separated from the presentation to ensure responsiveness. The compositing stream independently manages its memory to avoid garbage collection. It is implemented.
- Tile based rendering . We present the screen as a grid of tiles and draw each of them in parallel. In addition to benefiting from concurrency, tiles are also needed for performance on mobile devices. Partially implemented.
- Layered rendering . We split the display list into subtrees that can be processed by the GPU and render them in parallel. Partially implemented.
- Matching selectors . This task is surprisingly easy to parallelize. Unlike Gecko, Servo compares the selectors separately from the construction of the display tree, which is much easier to parallelize. It is implemented.
- Parallel placement . We build a mapping tree using a parallel DOM traversal that takes into account dependencies based on elements such as float. It is implemented.
- Forming the text . An important part of the in-line layout, text formation (using italics, bold type - approx. Per.) Is quite expensive and can potentially be parallelized. Not implemented.
- Parsing . We are currently writing a new HTML parser in Rust, focusing equally on security and spec compliance. In the process.
- Decoding images . Decoding multiple images in parallel is easy. It is implemented.
- Decoding of other resources . This is probably less important than image decoding, but everything that the page loads can be processed in parallel, for example, parsing style sheets or decoding video. Partially implemented.
- JS garbage collector competes with display . In almost any architecture that includes JS and mapping, JS will wait for the mapping to complete, often as often. This will be the most suitable time for garbage collection.
Difficulties
- Performance . Parallel algorithms, as a rule, require heavy compromises. It is important to really be quick. We need to make sure that Rust itself has performance similar to C ++.
- Data structures . Rust has a fairly innovative type system, in particular in order to make parallel types and algorithms safe, and we need to understand how to use it effectively.
- Immaturity of the tongue . The Rust compiler and language have recently stabilized. Rust also has fewer libraries than C ++; we can use libraries in C ++, but this requires more effort than just using header files.
- Non-parallel libraries . Some third-party libraries we need behave badly in a multi-threaded environment. In particular, there were difficulties with fonts. Even if libraries are technically and thread-safe, often this security is achieved through a single mutex library, which harms our parallelization capabilities.
Task architecture
Charts
Task Oversight Chart
Task Interaction Diagram
- Each rectangle represents a Rust task.
- Blue rectangles represent the main tasks from the browser pipeline.
- Gray rectangles represent auxiliary tasks for the pipeline.
- White rectangles represent work tasks. Each such rectangle represents several such tasks, the exact number of which depends on the workload.
- Dashed lines indicate the ratio of supervision.
- Solid lines indicate communication links.
Description
Each instance of constellation can be considered as a separate tab or window. It manages a task pipeline that accepts input, runs JavaScript for the DOM, arranges elements, builds display lists, renders display lists in tiles, and displays the final image to the surface.
The conveyor consists of four main tasks:
- Script - The main mission of the script is to create your own DOM and execute JavaScript. It receives events from a variety of sources, including navigation events, and redirects them as needed. When a task processing content needs to receive placement information, it should send a request to the placement task.
- The placement task - placement makes a DOM cast, computes the styles and builds the main structure of the display data - the display tree . The display tree is used to calculate the location of nodes and based on this, a display list is built , which is sent to the renderer.
- Renderer - The renderer receives the display list and renders the visible parts into one or more tiles, if possible, in parallel.
- The typesetter - the typesetter combines the tiles from the renderer and sends them to the screen. As part of the UI stream, the typesetter is the first to receive UI events that are usually immediately dispatched to the contents for processing (although some events, such as scroll events, may be pre-processed by the typesetter for responsiveness).
In this pipeline, two complex data structures are used in the interaction of tasks: the DOM and the display list. The DOM is passed from the content processing task to the placement processing task, and the display list from the placement task to the renderer. The search for an effective and type-safe way to represent, share and / or transfer these two structures is one of the main difficulties of this project.
nwrite DOM
Servo's DOM is a tree with support for node versioning, which can be shared between one writer and several readers. The DOM uses a copy-on-write strategy to allow the writer to modify the DOM in parallel with readers. The content processing task always writes, and the placement tasks or their subtasks are always read.
DOM nodes are Rust values whose lifetime is controlled by the JavaScript garbage collector. JavaScript accesses the DOM nodes directly - there is no XPCOM or similar infrastructure.
The DOM interface is currently not type-safe - it is possible to incorrectly manipulate nodes, which will lead to dereferencing of invalid pointers. The elimination of this insecurity is a high priority and necessary goal of the project; since DOM nodes have a complex life cycle this will lead to some difficulties.
Display list
Servo rendering is fully controlled by the display list, a sequence of high-level commands created by the layout task. Servo's display list is immutable, so it can be shared between competing renderers and it contains everything you need to display. This differs from the WebKit renderer, which does not use the display list and the Gecko renderer, which uses the display list, but also, during rendering, accesses additional information, for example, directly to the DOM.
JavaScript and DOM bindings
We currently use SpiderMonkey, although plug-in engines are a long-term low-priority task. Each content processing task gets its own JavaScript runtime. DOM bindings use the native engine API instead of XPCOM. Automatic generation of bindings from WebIDL in priority.
Multiprocess architecture
Just like Chromium and WebKit2, we intend to have a trusted process application and several less trusted process engines. The high-level API will, in fact, be based on IPC, and most likely with non-IPC implementations for testing and the single-process version, although it is assumed that the most serious users will use several processes. The engine process will use sandbox mechanisms provided by the operating system to limit access to system resources.
At the moment, we don’t intend to go into extremes with regard to the sandbox, like the Chromium developers, mainly because tying it to the sandbox requires a lot of work of developers (in particular on low-priority platforms like Windows XP or old Linux) and other aspects of the project are more priority . The Rust type system also adds an important layer of defense against memory protection vulnerabilities. This alone does not make the sandbox any less important in terms of protection against unsafe code, bugs in the type system and third-party libraries and libraries on the local computer, but this significantly reduces the possibility of attacks on Servo compared to other browser engines. In addition, we are concerned about performance related to some sandboxing techniques (e.g.
I / O and resource management
Web pages depend on many external resources, with a large number of mechanisms for receiving and decoding. These resources are cached at several levels - on disk, in memory, and / or in decoded form. Within a parallel browser, these resources should be distributed among competitive tasks.
Traditionally, browsers have been single-threaded, performing I / O on the “main thread”, where most of the calculations are also performed. This leads to problems with delays. Servo does not have a “main stream” and all external resources are loaded in a single resource manager .
Browsers use a lot of caches, and the task-based Servo architecture implies that it will probably use more caches than existing engines (we may have both a global cache based on a separate task and a local cache for the task , which stores the results from the global cache, to avoid calls through the scheduler). Servo should have a unified caching mechanism, with custom caches that will work in environments with a small amount of memory.
