Surrealism in JavaScript. NodeJS Development Tips

Hello, Habra!

Half a year ago, I thought: “Can I write a book?”, And I wrote it.



All documents are executed, pages are laid out, and circulation is printed. I will not beg your money on kickstarter or offer to buy something, but instead try to intrigue with development tips on NodeJS for public relations and to draw attention to the book.

Tip 1. SQL queries are best kept formatted.

It is better to store SQL queries formatted, because the code in this case is much easier to read and edit. Because SQL queries are usually quite long, it’s best to split them into several lines, and lines in JavaScript look best in an array.

Before:

var query = "SELECT g.id, g.title, g.description, g.link, g.icon, t.id as tag_id, c.title as comment_title FROM games AS g LEFT JOIN tags AS t ON t.game_id = g.id LEFT JOIN comments AS c ON c.game_id = g.id WHERE g.id = $1";

After:

var query = [
	"SELECT ",
	"    g.id, g.title, g.description, g.link, g.icon, ",
	"    t.id as tag_id, c.title as comment_title ",
	"  FROM games AS g ",
	"    LEFT JOIN tags AS t ON  t.game_id = g.id ",
	"    LEFT JOIN comments AS c ON  c.game_id = g.id ",
	"  WHERE ",
	"    g.id = $1"
];

Agree that in the second case, the request is much clearer for the reader. In addition, if you print a query to the console before a request to the database, then the array thrown in console.dir () is again much clearer than the string thrown in console.log ().

Tip 2. Life becomes easier when the API of various components accepts any format as input.

Suppose we have created a client for the database and want to execute a certain SQL query. At the input, we expect a string and parameters. Because using the last tip we decided to store long queries in arrays, we would like to forget about converting it to a string on each query.

Before:

dataBase(query.join(""), parameters, callback);

After:

dataBase(query, parameters, callback);

The code becomes simpler when the database query function (in this case dataBase) checks for itself in what form the request was sent to it, and if it is an array, it itself joins it with join ().

Tip 3. Color the console and format the output

Debugging programs on NodeJS is difficult because the standard developer console with all the chips, such as “breakpoints," is very lacking. All data is written to the console, and I want to make it more understandable. If your node is spinning somewhere on the server, and even in several instances, and even several different services on each instance hang, and you have access only via SSH, then the console can really make you suffer.

If in NodeJS to output to the console a line of the form "Hello world!" with ANSI control characters, it will be painted in different colors. An example of using ANSI control characters:



In order not to remember such hacks, you can connect the colors module and use the following syntax:

console.log("Error! Parameter ID not found.".red);

The string will be displayed in red. When developing with this module, you can colorize the messages in the console in various colors:

• Red (error).
• Yellow (warning).
• Green (all is well).
• Gray. They can display any parameters (for example, query parameters) that you can not pay attention to until you catch the error.

Console before color selection (when viewing quickly, information is perceived with difficulty):



Console after color selection (during quick viewing, information is perceived quite quickly):



Thanks to the colorized console, it will be much easier for you to monitor the server status. In addition, if you have several services hanging on one instance that work with different processes, you should also write separate methods in the modules for output to the console. For instance:

var book = {
    console: function(message, color) {
        console.log(("Book API: " + (message || ""))[(color || white")]);
    }
}

If all messages in the console are signed by the modules that send them, you can not only make a selection for a specific module, but also instantly find the source of the bug in a critical situation. Text formatting also simplifies the perception of information:



Thus, all the information in the console will be signed, and you can easily understand what events occur in certain modules. Again, color highlighting helps in stressful situations when a particular system fails and you need to urgently fix a mistake and you don’t have time to read logs (I do not urge you to debug production, it just happens). In my projects, I decided to rewrite the console module in order to be able to color not only strings, but also arrays and objects, as well as automatically sign all instances. Therefore, when connecting the module, I pass it the name of the package with which to sign messages. An example of using the new console module:

var console = require("./my_console")("Scoring SDK");
console.green("Request for DataBase.");
console.grey([
    "SELECT *",
    "  FROM \"ScoringSDK__game_list\"",
    "  WHERE key = $1;"
]);

An example of data output to the console:

Tip 4. Wrap all APIs in try / catch

We use the express framework at work. What was my surprise when I found out that the standard router object does not have a try / catch wrapper. For instance:

1. You wrote a curve module
2. Someone pulled it at the URL
3. Your server crashed
4. WTF !?

Therefore, always wrap the external module API in try / catch. If something goes wrong, your curved module will at least not overwhelm the whole system. The same situation on the client with the template “pick” (it is also called “listeners and publishers”). For instance:

• Module A posted a post.
• Modules B and C should hear it and respond.
• The system starts looping through subscribers in a loop.
• Module B crashes with an error.
• The cycle breaks off and the callback function of module B is not called.

It is much better to do a try / catch search and if module B really crashes with an error, then at least it won’t kill the system and module C will do its job after hearing the event.

Because when writing API modules, I had to separate private and public methods again and again, and after wrapping all public methods in try / catch, I decided to automate this business and wrote a small module for API auto-generation. For example, we throw an object of the form into it:

var a = {
	_b: function() { ... },
	_c: function() { ... },
	d:  function() { ... }
}

It is clear from the naming of the methods that the first two are private, and the last is public. The module will create a wrapper to call the last one, of the form:

var api = {
	d: function() {
		try {
			return a.d();
		} catch(e) {
			return false;
		}
	}
};

Thus, I began to generate a wrapper for the API of all modules, which in case of an error did not pass it further. This made the code more stable as an error of an individual developer, merged into production, could no longer drop the entire server with all its functionality.

API generation example:

var a = {
	_b: function() { ... },
	_c: function() { ... },
	d:  function() { ... }
}
var api = api.create(a);
api.d(); // пример вызова

Tip 5. Collect requests in configs

I think every web developer had a situation where there was some greasy client who needed a small API to work with the database on the server. A couple of requests to read, a couple to write, and a few more to delete information. There is usually no logic in such a server, and it is just a collection of queries.

In order not to write wrappers for such operations each time, I decided to make all requests in JSON, and design the server as a small module that provides me with an API for working with this JSON.

An example of such a module under express:

var fastQuery = require("./fastQuery"),
	API = fastQuery({
		scoring: {
			get: {
				query: "SELECT * FROM score LIMIT $1, $2;"
				parameters: [ "limit", "offset" ]
			},
			set: {
				query: "INSERT INTO score (user_id, score, date) VALUES ...",
				parameters: [ "id", "score" ]
			}
		},
		profile: {
			get: {
				query: "SELECT * FROM users WHERE id = $1;",
				parameters: [ "id" ]
			}
		}
	});

You probably already guessed that the module would run through JSON and look for objects with query and parameters properties. If such objects are found, then he will create a function for them that will check the parameters, go to the database with requests, and send the result to the client. At the output, we get the following API:

API.scoring.get();
API.scoring.set();
API.profile.get();

And we’ll already attach it to the router object:

exports.initRoutes = function (app) {
    app.get("/scoring/get", API.scoring.get);
    app.put("/scoring/set", API.scoring.set);
    app.get("/profile/get", API.profile.get);
}

I will not run my framework for this purpose, as the technology stack on the server is different in different companies. To work with such an object, in any case, you will need to write a small binding, on top of something, to process requests and work with the database. In addition, perhaps at the time you read these lines, there will already be several ready-made frameworks for this task.

Now imagine that you have two more server developers. One writes in PHP, and the second in Java. If you have all the server logic limited to only such JSON with a list of database queries, then you can instantly transfer / deploy a similar API not only to another machine, but also in a completely different language (provided that communication with the client is standardized and that’s all communicate via REST API).

Tip 6. Take everything out in configs

Because I like to write configs, I have repeatedly had a situation when the system has standard settings, settings for a specific case and settings that occurred at a given point in time. I had to make mix of different JSON objects. I decided to allocate a separate module for these purposes, and at the same time added the ability to take JSON objects from the file, because storing settings in a separate json file is also very convenient. Thus, now that I need to set preferences for something, I write:

var data = config.get("config.json", "save.json", {
	name: "Petr",
	age: 12
});

As you might have guessed, the module iterates over the arguments passed to it. If this is a line - then he tries to open the file and read the settings from it, if it is an object - then he immediately tries to cross it with the previous ones.

In the example above, we first take some standard settings from the config.json file, then apply the saved settings from the save.json file to them, and then add the settings that are relevant at the given time. At the output, we get a mix of three JSON objects. The number of arguments passed to the module can be any. For example, we can only ask for the default settings:

var data = config.get("config.json");

Tip 7. Working with files and Social Link module for CEO

One of the main features that I like about NodeJS is the ability to work with files and write parsers in JavaScript. At the same time, the NodeJS API provides many methods and methods for solving problems, but in practice, not much is needed. For half a year of active work with parsers, I used only two commands - read and write to a file. Moreover, in order not to suffer with callback-functions and various problems of asynchrony, I always did all the work with files in synchronous mode. So there was a small module for working with files, whose API was very similar to localStorage:

var file = requery("./utils.file.js"), // подключили модуль
	text = file.get("text.txt);        // прочитали текст в файле
file.set("text.txt", "Hello world!");  // записали текст в файл

On the basis of this module for working with files, other modules began to appear. For example, a module for CEO. In one of the previous articles, I already wrote that there are a huge number of different meta tags related to CEO. When I started writing the assembly system for HTML applications, I paid special attention to the CEO.

The bottom line is that we have a small text file with a description of the site / application / game and the HTML file itself for parsing. The Social Link module should find all meta tags related to the CEO in the HTML file and fill them out. The external module API expects text from a file as input. This is done in order to be able to connect it to the assembly systems and run through it the text of several files without each time causing an extra read / write to the file.

For example, before the module:

After the module:

The list and description of all meta tags for CEO and not only, you can see in the book http://bakhirev.biz/ .

Tip 8. Life is easier without callbacks.

Many developers complain about endless chains of callbacks when writing a server on NodeJS. In fact, you are not always obliged to write them, and you can often build an architecture in which such chains will be minimal. Consider a small task.

Task:
Transfer files from server A to server B, get some information from the database, update this information, send data to server B.

Solution:
This is a rather routine procedure that I have repeatedly had to perform to solve any sorting / processing tasks. Typically, developers create a loop and some kind of callback function with the nextFile () method. When at the next iteration we call nextFile (), the callback mechanism starts from the beginning, and we process the next file. As a rule, it is required to process only one file at a time and, upon successful completion of the procedure, proceed to the processing of the next file. To simplify nesting, a code of the form will help us:

var locked = false,
	index = 0,
	timer = setInterval(function() {
		if(locked) return;
		locked = true;
		nextFile(index++);
	}, 1000);

Now we will try to start processing the file once a second. If the program is freed, it will set locked to false and we can run the next file for processing. Such constructions very often help reduce nesting, distribute the load over time (since the next iteration of processing starts no more than once per second) and at least crawl a bit from endless callbacks.

Total
Files with modules can be downloaded here: http://bakhirev.biz/_node.zip (now 2 a.m. and I’m too lazy to deal with GitHub and bring the code into a human form).
The book is here: http://bakhirev.biz/
In case of a habro-effect here in PDF.

If you liked the tips above, I want to warn you right away that the book is about something else. And at the end there is a list of different smart people who made an invaluable contribution themselves without suspecting it, and who definitely should be found and read separately.