June 2, 2019 at 14:09

Best Practices Node.js - Project Structure Tips


Hello, Habr! I present to you the adapted translation of the first chapter of " Node.js Best Practices " by Yoni Goldberg. A selection of recommendations on Node.js is posted on github, has almost 30 tons of stars, but so far has not been mentioned on Habré. I suppose that this information will be useful, at least for beginners.

1. Project structure tips


1.1 Structure your project by component


The worst mistake of large applications is the monolith architecture in the form of a huge code base with a large number of dependencies (spaghetti code), this structure greatly slows down the development, especially the introduction of new functions. Tip - separate your code into separate components, for each component, select your own folder for the component modules. It is important that each module remains small and simple. In the "Details" section, you can see examples of the correct structure of projects.

Otherwise: it will be difficult for developers to develop the product - adding new functionality and making changes to the code will be slow and have a high chance of breaking other dependent components. It is believed that if the business units are not divided, then there may be problems with scaling the application.

detailed information
One paragraph explanation

For applications of medium size and above, monoliths are really bad - one large program with many dependencies is simply difficult to understand, and it often leads to spaghetti code. Even experienced programmers who know how to properly “prepare modules” spend a lot of effort on architectural design and try to carefully evaluate the consequences of each change in the relationships between objects. The best option is an architecture based on a set of small component programs: divide the program into separate components that do not share their files with anyone, each component should consist of a small number of modules (for example, modules: API, service, database access, testing etc.), so that the structure and composition of the component are obvious. Some may call this architecture “microservice”, but it’s important to understand that microservices are not a specification that you should follow, but rather a set of some principles. At your request, you can adopt both individual of these principles and all the principles of microservice architecture. Both methods are good if you keep the code complexity low.

The least you have to do is define the boundaries between the components: assign a folder in the root of your project for each of them and make them standalone. Access to the component functionality should be implemented only through a public interface or API. This is the basis for keeping the simplicity of your components, avoiding the “hell of dependencies” and letting your application grow to a full-fledged microservice.

Quote from the blog: “Scaling requires scaling the entire application”
From MartinFowler.com Blog
Monolithic applications can be successful, but people are increasingly frustrated with them, especially when thinking about deploying to the cloud. Any, even small, changes in the application require the assembly and redeployment of the entire monolith. It is often difficult to constantly maintain a good modular structure in which changes in one module do not affect others. Scaling requires scaling the entire application, and not just its individual parts, of course, this approach requires more effort.

Quote from the blog: “What is the architecture of your application talking about?”
From the uncle-bob blog
... if you have been to the library, then you represent its architecture: the main entrance, reception desks, reading rooms, conference rooms and many halls with bookshelves. Architecture itself will say: this building is a library.

So what is the architecture of your application talking about? When you look at the top-level directory structure and the module files in them, they say: I am an online store, I am an accountant, I am a production management system? Or are they shouting: I'm Rails, I'm Spring / Hibernate, I'm ASP?
(Translator's note, Rails, Spring / Hibernate, ASP are frameworks and web technologies).

Correct project structure with stand-alone components



Incorrect project structure with grouping files according to their purpose



1.2 Separate the layers of your components and do not mix them with the Express data structure


Each of your components should have “layers”, for example, to work with the web, business logic, access to the database, these layers should have their own data format not mixed with the data format of third-party libraries. This not only clearly separates the problems, but also greatly facilitates the verification and testing of the system. Often, API developers mix layers by passing Express web layer objects (like req, res) to the business logic and data layer - this makes your application dependent and highly connected with Express.

Otherwise: for an application in which layer objects are mixed, it is more difficult to provide code testing, organization of CRON tasks and other non-Express calls.

detailed information
Divide the component code into layers: web, services and DAL The



reverse side is the mixing of layers in one gif-animation



1.3 Wrap your basic utilities in npm packages


In a large application consisting of various services with their own repositories, such universal utilities as a logger, encryption, etc., should be wrapped with your own code and presented as private npm packages. This allows you to share them between several codebases and projects.

Otherwise: you have to invent your own bike to share this code between separate code bases.

detailed information
One paragraph explanation

As soon as the project begins to grow and you have different components on different servers using the same utilities, you should start managing dependencies. How can I allow multiple components to use it without duplicating the code of your utility between repositories? There is a special tool for this, and it is called npm .... Start by wrapping third-party utility packages with your own code so that you can easily replace it in the future, and publish this code as a private npm package. Now your entire code base can import the utilities code and use all the npm dependency management features. Remember that there are the following ways to publish npm packages for personal use without opening them for public access: private modules ,private registry or local npm packages .

Sharing your own shared utilities in different environments


1.4 Separate Express into “application” and “server”


Avoid the unpleasant habit of defining your entire Express application in one huge file, split your 'Express' code into at least two files: an API declaration (app.js) and a www server code. For an even better structure, place an API declaration in the component modules.

Otherwise: your API will only be available for testing through HTTP calls (which is slower and much more difficult to generate coverage reports). Still, I suppose, it’s not too much fun working with hundreds of lines of code in one file.

detailed information
One paragraph explanation We

recommend using the Express application generator and its approach to building the application base: the API declaration is separated from the server configuration (port data, protocol, etc.). This allows you to test the API without making network calls, which speeds up testing and makes it easier to get code coverage metrics. It also allows you to flexibly deploy the same API for different server network settings. As a bonus, you also get a better separation of responsibilities and a cleaner code.

Sample code: API declaration, must be in app.js
var app = express();
app.use(bodyParser.json());
app.use("/api/events", events.API);
app.use("/api/forms", forms);

Code example: server network parameters, should be in / bin / www
var app = require('../app');
var http = require('http');
/**
 * Получаем порт из переменных окружения и используем его в Express.
 */
var port = normalizePort(process.env.PORT || '3000');
app.set('port', port);
/**
 * Создать HTTP-сервер.
 */
var server = http.createServer(app);


Example: testing our API using supertest (a popular testing package)
const app = express();
app.get('/user', function(req, res) {
  res.status(200).json({ name: 'tobi' });
});
request(app)
  .get('/user')
  .expect('Content-Type', /json/)
  .expect('Content-Length', '15')
  .expect(200)
  .end(function(err, res) {
    if (err) throw err;
  });


1.5 Use a secure hierarchical configuration based on environment variables


An ideal configuration setting should provide:

(1) reading keys from both the configuration file and environment variables,
(2) keeping secrets outside the repository code,
(3) hierarchical (rather than flat) data structure of the configuration file to facilitate work with settings.

There are several packages that can help implement these points, such as: rc, nconf, and config.

Otherwise: failure to comply with these configuration requirements will lead to disruption of the work of both an individual developer and the entire team.

detailed information
One paragraph explanation

When you are dealing with configuration settings, many things can annoy and slow down the work:

1. Setting all parameters using environment variables becomes very tedious if you need to enter 100+ keys (instead of just fixing them in the configuration file), however if the configuration will be specified only in the settings files, this may be inconvenient for DevOps. A reliable configuration solution should combine both methods: both configuration files and parameter overrides from environment variables.

2. If the configuration file is “flat” JSON (that is, all keys are written as a single list), then with an increase in the number of settings it will be difficult to work with it. This problem can be solved by forming nested structures containing groups of keys according to settings sections, i.e. organize a hierarchical JSON data structure (see example below). There are libraries that allow you to store this configuration in several files and combine the data from them at run time.

3. It is not recommended to store confidential information (such as a database password) in configuration files, but there is no definite convenient solution for where and how to store such information. Some configuration libraries allow you to encrypt configuration files, others encrypt these entries during git commits, or you don’t have to save secret parameters in files at all and set their values ​​during deployment via environment variables.

4. Some advanced configuration scenarios require you to enter keys through the command line (vargs) or synchronize configuration data through a centralized cache such as Redis so that multiple servers use the same data.

There are npm libraries that will help you with the implementation of most of these recommendations, we recommend that you take a look at the following libraries:rc , nconf and config .

Code example: hierarchical structure helps to find records and work with voluminous configuration files

{
  // Customer module configs 
  "Customer": {
    "dbConfig": {
      "host": "localhost",
      "port": 5984,
      "dbName": "customers"
    },
    "credit": {
      "initialLimit": 100,
      // Set low for development 
      "initialDays": 1
    }
  }
}

(Translator’s note, you can’t use comments in the classic JSON file. The above example is taken from the config library documentation, which adds functionality for pre-clearing JSON files from comments. Therefore, the example is quite working, but linters like ESLint with default settings may "Swear" on a similar format).

Afterword from the translator:

  1. The description of the project says that the translation into Russian has already been launched, but I did not find this translation there, so I took up the article.
  2. If the translation seems very brief to you, then try to expand the detailed information in each section.
  3. Sorry that the illustrations were not translated.
Tags:

Also popular now: