Dynamic display of logs in the browser on Node.js & WebSocket

Dynamic display of logs in the browser on Node.js & WebSocket

Sometimes there is a need to dynamically monitor the appearance of new files in certain folders, as well as display the contents of the log files in a browser window. With this simplified example, I want to show how this can be done.

Tools and modules for the solution

Node.js. was selected for the server side. But a similar example can be similarly implemented using EM in rubles. In this case, the Socket.IO library was bribed , as well as several simple modules for working with files.



Utilities for watching file trees in node.js - the meaning of which is to track the appearance, deletion, as well as changes in the files inside the specified folder.

An example of tracking the appearance / deletion of files inside common.NODES_PATH and sending a message about these events.

watch.createMonitor(common.NODES_PATH, function (monitor) {
  monitor.on("created", function (f, stat) {
    var file_name = common.removeRoot(f); 
    console.log(file_name + ' created');
    var idx = log_files.indexOf(file_name);
    if (idx < 0) {
      log_files.push(file_name);
      io.sockets.emit('log_files_add', [file_name]);
      watchFile(file_name);
    }
  })
  monitor.on("removed", function (f, stat) {
    var file_name = common.removeRoot(f); 
    console.log(file_name + ' removed');
    var idx = log_files.indexOf(file_name);
    if (idx >= 0) {
      log_files.splice(idx, 1);
      io.sockets.emit('log_files_remove', [file_name]);
      unwatchFile(file_name);
    }
  })
})

A nodejs directory walker is a package that allows you to walk through the insides of a specified folder and receive events on finding a particular file type. Or, if simplified, then just recursively go through all the subfolders and see what lies there. (see example below)

Jade - Template engine - to quickly outline a simplified UI and test the idea.

Implementation example

• We create a web server on a given port;
• When connecting to the server for the first time, we walk through the folders using the walker and send the initial portion of information about existing files (plus filter by extension) and the last 20 lines.
  
  

  io.sockets.on('connection', function(socket) {
    var walk = require('walk');
    // receive list of log files
    var walker  = walk.walk(common.NODES_PATH, { followLinks: false });
    walker.on('file', function(root, stat, next) {
      if(/log$/.test(stat.name)) {
        var file_name = common.removeRoot(root + '/' + stat.name);
        var idx = log_files.indexOf(file_name);
        if (idx < 0) {
          log_files.push(file_name);
          // send initial data portion
          var tail = spawn('tail', ['-n', 20].concat(common.NODES_PATH + '/' + file_name));
          tail.stdout.on('data', function(data) {
            console.log('emit starting ' + file_name + ':lines' + ' with ' + data.toString('utf-8'));
            io.sockets.emit(file_name + ':lines', data.toString('utf-8').split('\n'));
          });
          watchFile(file_name);
        }
      }
      next();
    });
    walker.on('end', function() {
      socket.emit('log_files_add', log_files);
    });    
  })
  

  
  
  
• Using watch, we create a monitor that will monitor the appearance of new files or deletion. (When deleting a file, the block displaying the given log is deleted accordingly)
  
  

  function watchFile(file) {
    console.log('start watching: ' + file);
    var tail = spawn('tail', ['-f'].concat(common.NODES_PATH + '/' + file));
    watching_processes[file] = tail;
    tail.stdout.on('data', function(data) {
      console.log('emit ' + file + ':lines' + ' with ' + data.toString('utf-8'));
      io.sockets.emit(file + ':lines', data.toString('utf-8').split('\n'));
    })
  }
  function unwatchFile(file) {
     console.log('end watching: ' + file);
     var process = watching_processes[file];
     process.kill();
     watching_processes[file] = null;
  }
  

  
  
  

Launch

In common.js, the settings for which folder to monitor.
var NODES_PATH = pathResolver.resolve(__dirname + '/../nodes');

When * .log appears inside folders and subfolders, they will be displayed in the browser window

npm installfor installing packages to start the server.

[~/Projects/autotest_node] git:master $ mkdir nodes
[~/Projects/autotest_node] git:master $ mkdir nodes/test1
[~/Projects/autotest_node] git:master $ mkdir nodes/test2
[~/Projects/autotest_node] git:master $ echo "test_message1" >> nodes/test1/first.log
[~/Projects/autotest_node] git:master $ echo "test_message1" >> nodes/test2/first.log
[~/Projects/autotest_node] git:master $ echo "test_message2" >> nodes/test2/first.log
[~/Projects/autotest_node] git:master $ echo "test_message2" >> nodes/test1/first.log

npm start

Source

https://github.com/catz/autotest_node

Conclusion

This example can be useful, for example, if you want to share the results of the execution of automatic test cases and give the opportunity to monitor the result of their execution to others. Although there are more advanced tools for such tasks, but if you want to make a simple tail -f in the browser, then a similar example may well work.