Erlang in Risovask, part 2 - answers to questions

This article is a continuation of a series of articles about the Erlang / Erlang programming language and its use in our Risovaska project . First of all, I want to answer the questions asked in the comments to the first article , but many questions were asked. So let's continue:

Install Erlang on Windows

Installing Erlang is quite simple. First, download the distribution from the official site . We need the latest version of R12B-5, the Windows binary column (incl. Documentation). After installing “Start” - “All Programs” in me, the Erlang OTP R12B item will appear and the Erlang sub-item in it. If you select it, then the nameless node starts up along with a shell in which it is already possible to execute Erlangian code, including from my examples. I will adhere to the name “node”, since the official name in Russian has not yet been established, although the word “node” is translated as “node”.

Here are links to install Erlang for Mac OS X and Linux .

To complete the installation, you need to write the following variables into the Windows environment variables:
1. Path to the directory where Erlang was installed:
ERL_TOP = C: \ Program Files \ erl5.6.5
the same path, but with the addition of \ bin , ie C: \ Program Files \ erl5.6.5 \ bin must also be written in the PATH variable if it is not already there

2. The HOME variable, if it is not already registered with you, for example:
HOME = C: \ Documents and Settings \ tolik

3. I recommend immediately registering the path to third-party Erlang libraries:
ERL_LIBS = C: \ Work \ erl_libs
In the future, put all third-party libraries that are not included in the Erlang / OTP package into this directory. This will make it easy to install the new version of Erlang / OTP, deleting the old one without rearranging third-party libraries. By the way, new versions of Erlang / OTP are released twice a year with a regularity.

Lastly, in the HOME directory, create the file ".erlang.cookie". This is a regular text file that stores a text cookie in one line, for example: "AMMVFJKDTENSFHDJDGDERSW". This cookie will be used when nodes interact with each other. The fact is that two nodes can interact with each other only if they have the same cookie's. If you are not interested in the interaction of several nodes with each other, skip this step.

In the same HOME directory, it is convenient to create a ".erlang" file in which you can write code on Erlang, which will be automatically executed when any node starts on this computer. Typically, code is written to this file adding the path to the executable files of your application to the search list:

code:add_pathz("Путь к папке ebin вашего приложения").

Now you can start the node directly from the command line simply by typing: “erl”. You can run a node without a shell or as a daemon. If you started a node without a shell, you can connect to it by launching the second node (you can also use it on another computer) by doing:

erl -name test@192.168.0.11 -remsh main@192.168.0.11

where main@192.168.0.11 is the name of the node we are connecting to, and test@192.168.0.11 is the name of the new node. After connecting, you can execute all the commands as if you are on the first node.

The launch parameters are described in more detail in the erl module .

Emacs as a development environment

Until recently, almost the only convenient way to write and debug Erlang programs was Emacs (mistakenly mistaken for a regular text editor) in conjunction with Distel . Fairly good instructions for setting up and working with both packages in Russian can be found here . In our work, we still use them, although they look like ancient dinosaurs, compared to modern development environments.

New development environment: erlIDE

To replace Emacs, a new erlIDE development environment is being developed. The current version 0.4.3 is already quite stable and does not fall, like earlier versions. It contains almost all the usual tools for modern IDEs: a debugger, a context help, simple code navigation, syntax highlighting, etc. Installation instructions are written in sufficient detail and usually do not raise questions. erlIDE works as a plug-in for Eclipse , so you need to first install Java on your computer (preferably at least version 6), and then Eclipse . Eclipse IDE for Java Developers (85 MB) will be enough for work. On Windows, I would recommend starting with erlIDE right away so you don't get used to Emacs.

Now back to the language itself.

Create the first application

To get started, create a folder where you will have the project. Inside it, you need to create the following subfolders:

• ebin - the compiled code (.beam) will be put here
• src - application sources (.erl)
• include - header files (.hrl)
• doc - documentation
• priv - everything else the application needs at work, but it is not included in other folders

This is the standard application structure recommended by OTP. In the application root folder, create an emakefile with the following contents:

{"src/*", [debug_info, {i, "include"}, {outdir, "ebin"}]}.

and then in the node shell, being in the application root folder, you can simply execute the command: make: all () to recompile all the application modules.

In the “ebin” subfolder you need to create your_application.app file with approximately the following contents: And lastly , in the “src” folder you need to create the main project file: you_application.erl with the minimum contents: Thus, we created a minimal application on Erlang, which has nothing yet does. Also, the main application supervisor has not yet been implemented (I will write about it later). For more information on creating an application, see the OTP documentation.
{application, your_application.app,
[{description, "Test application"},
{vsn, "1"},
{modules, [you_application, you_application_sup]},
{registered, []},
{applications, [kernel, stdlib, sasl]},
{mod, {your_application,[]}}
]}.


-module(you_application).
-behaviour(application).

-export([start/2, stop/1]).

start(_Type, _Args) ->
your_application_sup:start_link().

stop(_State) ->
ok.

Hot swap code

A very useful feature of Erlang / OTP is the hot swapping of code without stopping the system. How does this work on the example of one module? For example, you changed the code in one of the application modules and want to replace it on the fly in a running system. In this situation, in Erlang it is not necessary to stop the entire application. It is enough to type in the node shell: l(your_module).and after the execution of this command, the new code will work. In more detail, the Erlang virtual machine stores two copies of each module in memory: the current and the previous one. Why keep the previous version? In order to quickly roll back to the previous version in case of incorrect operation of the new code without stopping the application.

Of course, you can replace on the fly not only a separate module, but the entire application. For this:

• we create an Application Upgrade File, in which we prescribe the rules for replacing the old version with the new one, as well as the dependencies between the modules during the version replacement
• create a release upgrade file
• install the new version of the application on top of the old

To reduce the size of the article, I send it to the documentation for more information , where these steps are described in more detail.

But what to do when you have a whole distributed system that consists of several servers and you need to replace all of them at the same time without stopping the entire system. Unfortunately, Erlang / OTP does not offer a ready-made convenient solution here. However, OTP includes erl_boot_server, which allows you to load application code at startup, including configuration files from another node. Then, overloading the entire system can be implemented as follows: first, the code is hot-swapped on the main server, the code from which the other nodes were loaded at startup, using the erl_boot_server module. And then a message is sent to all the nodes to overload the nodes. Upon receipt of such a message, the init command is executed on the node: restart / 0 overloading the node without stopping it completely. After overload, all nodes take the already updated code from the main server. Approximately this approach is described in two articles Setting up Erlang on Amazon EC2 (second part of the article) and Upgrading your Erlang cluster on Amazon EC2 .

Behaviours

Another interesting feature of the language is behaviours, which are analogues of interfaces in traditional languages, and more precisely abstract classes. If, for example, it -behaviour(application).is written at the beginning of the module, this means two things: part of the logic is already implemented in OTP and the second, a certain set of callback functions must be implemented in the module, without which the module will not compile. In our case, we must implement the start and stop functions. For more information, send to the documentation on behaviours .

Let the process die

As I already noted in my first article, the high reliability of systems built on Erlang is not based on the fact that all the code is written so that it does not crash, but so that if there is any error in the workflow, there is always someone watching it process that will correct the situation and restore the system. Indeed, most often the workflow usually does not know exactly what to do in case of an error, therefore it is more correct to simply die and let the observing process understand the situation. It is done like this:

• firstly, the observing process executes the command famous in Erlang:
  process_flag(trap_exit, true).
  This means that this process catches messages about the fall of the processes associated with it. After executing this command, if the workflow crashes, then a message of the form arrives in the observing process: {'EXIT', From, Reason}It is important that without executing this command, in the event of an error in the workflow, all the processes associated with it will fall along the chain until the process encounters it, which executed process_flag (trap_exit, true).
• secondly, the observing process creates a workflow associated with it:
  spawn_link(?MODULE, worked_function, [Arguments]).
• and thirdly, the observing process begins to listen to messages:
  receive
{'EXIT', From, Reason} ->
supervising_function() % выполнить саму себя (см. ниже)
end

  

In total, the minimum code of the observing process will look like this:
supervising_function() ->
process_flag(trap_exit, true).
spawn_link(?MODULE, worked_function, [Arguments]).
receive
{'EXIT', From, Reason} ->
supervising_function()
end
end.

Building Bricks Erlang / OTP

But it’s not at all necessary to write observing processes from scratch, processes of access to shared resources, etc. For less than twenty years of language development in Erlang / OTP, a reliable and debugged set of basic components has been developed on which an application written in Erlang is usually built. Here they are:

• gen_server - the basic server implementation that supports client-server interaction in the system. I note that here the concept of "server" does not coincide with the generally accepted concept, but means a specific Erlang / OTP server. Typically, in a system, he is responsible for accessing the system’s shared resources.
  
• gen_fsm - the base module that implements the state machine
• gen_event - the base module that implements the event handling functionality
• and, of course, supervisor - the module is designed to start, stop, and monitor its child processes. In order not to write a lot of theory, I will give a simple example of the supervisor initialization function: This code will start one child process, the maximum number of child process restarts = 10 with a frequency of no more than 3600 ms.
  init() ->
RestartStrategy = one_for_one
MaxRestarts = 10,
MaxTimeBetRestarts = 3600,
SupFlags = {RestartStrategy, MaxRestarts, MaxTimeBetRestarts},
ChildSpecs = [{Some_Module, {Some_Module, Function, []}, permanent, TimeOut,
worker, % тип дочернего процесса
[Some_Module]}],
{ok, {SupFlags, ChildSpecs}}.
  

I note that in all the basic components described above (except for the supervisor), there is a code_change function that will automatically execute when the application is hot reloaded. In this function, you can write code that provides a smooth transition from one version of the application to another.

Unfortunately, a detailed description of each module and the relationships between them will take several articles, moreover, they are well described in Design Principles . And you don’t have to go far for examples of use, take any fairly well-known application or library written in Erlang and see. Almost all of them are written using these basic components.

At the end of the article I want to give a link to a good Wikipedia article about Erlang and two good ones ( 1 ,2 ) a selection of resources on Erlag, which was collected by Dima Smolin. For those who are interested in the Erlang language, these links are enough to dive deep into the language.

To be continued...

As in the first article, the list of topics to consider remains the same: using the distributed Mnesia database (which is part of Erlang / OTP), using Amazon S3 and Amazon EC2 with examples of using these technologies on Risovaska servers .