June 16, 2014 at 12:16Two-pane web file manager Cloud Commander
There are many file managers, but there is one that I think will be interesting for many to learn. After all, it is two-panel, works in a browser, is equipped with an editor (with syntax highlighting) and a console, consists of a client and a server, and is written in JavaScript / Node.js.
Website: cloudcmd.io
Demo: heroku , jitsu
The first computer book I read was Windows: Wizard Lab . She talks about a variety of utilities for Windows 9x. Some of them no longer exist (Zip Magic 2000, for example), while others are actively used, and most importantly, are developed to this day (Total Commander). Most of all I liked the section about file managers. I didn’t have a computer yet, but already then I realized that it wasn’t serious to use Explorer, and it’s much more correct and convenient to use Two-Panel File Managers. I tried everything that was in the book at every computer that I managed to visit. Most of all, of course, I liked Total Commander. He is the best in his business, it is indisputable.
A few years later, I got a computer. After some time, next to Windows, I installed Linux, and wanted to find something suitable for convenient file management. I didn’t really succeed. Yes, Midnight Commander in * nix is the best, that's true. But many of the functions that I was so used to using Total, were not in it. They were not in the graphic managers. One of these functions is moving the pointer of the current file while entering the name (when there are a lot of folders, and I have a lot of music - flipping through the list is not the most pleasant of occupations).
In the end, I was more pleasant to listen to music, watch videos, and manage files in a bare console. Which I stopped for a while.
A few years later, getting a job in a small company, I realized that getting into my computer would be much less likely. And indeed, the circumstances are such that more often I work for other people's computers. And since getting used to the new is not very easy for me, I began to increasingly use languages and development tools that work in the browser and do not require installation, configuration, or other lengthy things. I started using Cloud9 , Koding and, of course, GitHub .
I got excited about the idea of cloud services, I liked the openness and capabilities of these projects so much that I began to make my own. This is the file manager of Cloud Commander.
Most analogues are written in php, which is not very close to me. Most of the code I write in JavaScript, Therefore, accordingly, I would like to see more applications written in a language that may not be the best, but the most common, that's for sure.
There are a lot of file managers for the web. But, practically, each of them has several fundamental problems:
Of the good managers, I can cite, for example, the following:
The applications described above do their job very well. But from the usual file manager, for example, I expect another. And if there is a need to work in a browser - why not make programs of this type more friendly, not only for ordinary users, but also for more advanced ones who spend most of their time in classic two-panel file managers.
I couldn’t find sane two-panel file managers that would work in a browser 2 years ago, when I started writing my implementation, I can’t find it now. Although I regularly monitor this issue.
At first, I had ideas to continue someone’s undertakings, to connect to the project, so to speak. I found an analogue (unfortunately I can’t remember the link) total commander on SourceForge, with a web interface, it was written in PHP, with which I was playing at that time. All of its code was combined into one file, and I didn’t really want to deal with this. In addition, in those days, I began to be actively interested in Node.js, and I wanted to try it on a new project.
Of the interesting things that have appeared since that time, I can only say that Metro Commander. True, there is no chance to feel it, since it works only on win8, and not in the browser, but natively. But the screenshots look impressive. Otherwise, there are no special movements in this direction, at least in the places where I am looking.
And so, what makes Cloud Commander so interesting that it is worth paying attention to? Let us examine these points in more detail since it is really important.
Of the most interesting features it is worth highlighting the following:
The main difference is the two-panel interface with the usual keyboard shortcuts. But that's not all. In contrast to other browser file managers, Cloud Commander has a console and editor.
The editor supports syntax highlighting in more than 110 languages. The file format is determined by its extension. The source code for Cloud Commander is corrected in it itself (and this article is written in it too).
The console allows you to execute commands on the server, and it does not matter whether it is Linux, Mac OS or Windows. The server on which Cloud Commander is running is controlled from the console directly in the browser.
An important difference is also Node.js as the platform on which the File Manager runs. No more php + apache.
The presence of advantages also implies the presence of disadvantages, without this in any way. The most significant:
These are fundamental flaws. If we talk about pressing issues, that is, such. In the editor:
As mentioned above, Cloud Commander consists of a client and a server. A little has already been said about the client, so we will probably start from the server.
The server can work without the installed dependencies, while it goes into limited operation mode. In this mode, the console does not work, js / css / html optimizations are not performed, copying, moving and deleting folders does not work. After installing the modules prescribed in package.json, the following are used:
Some modules work both on the client and on the server, such is diff-match-patch , developed for a long time, but it works very stable.
On the client, the list of modules used is much wider. It:
Most of the questions are probably caused by the archiver on the client. It is used to reduce the size of data sent to the server by the editor. This mode can be turned on (and off) in the settings. In fact, as I said, the bottleneck in the client-server application is data transfer. Packing (and unpacking on the server), in turn, is extremely fast.
But the text does not have to always be sent entirely. It is advantageous to send to the server only the data that has changed, that is, the patch. Therefore, if, between edits, the file on the server did not change - a patch is sent and applied on the server (provided that the file does not weigh very much, because each such operation is to load the entire file into RAM), if changed, the compressed data is sent . Due to this, the data processing speed is very close to the desktop application.
Downloading data from the server is also not needed every time you open files. Therefore, when opened (files) are placed in localStorage, in a place with a sha-1 hash. And, if the hash has changed (without our knowledge), the file is downloaded again, otherwise the hash is updated every time the file is saved. The same is with directories. If the option is enabled, the contents of the directory are loaded once, and to update it, press Ctrl + R (or delete / create a new file / folder).
In such a large application with a huge number of modules, it would be very difficult to figure out if all the files were loaded immediately, when the page loaded. Such an application would be difficult to maintain, as some programs are updated from time to time, while others become obsolete and require replacement. Therefore, the application is divided into modules: client and server.
Client modules are loaded as needed. If a person needs a menu, it loads, if the console didn’t come in handy for the entire session, its files did not even load and did not start executing, which significantly increases the speed of work and saves resources both on the client and on the server.
Threads node.js is a very powerful tool that is fundamentally different from what other scripting languages have. In the process of immersion in node.js, the idea that you can combine files into a stream and give it as if the file is one, did not leave me. I thought there would be delays in speed, but no. Everything works like a clock, and there is no particular slowdown, but instead, it becomes possible not to combine files into one, and not load them sequentially, but to load them as one file .
This idea, recently, began to promote in jsDelivr . And, it seems to me, this is the right direction.
In short: if you need to download the jquery.js and jquery.fancybox.js file, you can do it this way:
Using the “:” symbol, file names are separated from each other, so you can combine absolutely anything, and this should not particularly affect the speed of the server, since the files are read sequentially, but immediately after reading they are given to the client.
If the file cannot be read, an error message is sent instead, after which the stream closes.
Enough has been said about the application itself, but there are a few things that I would like to say about development. As already mentioned, Cloud Commander is written in itself.
The project is hosted on a github . It has two branches: dev and master .
In the first one there is an option that is being developed, all the changes are made there, and although ideally everything should always work, sometimes different incidents happen.
In the second branch is the latest stable version. It can always be taken from the repository, everything should be fine with it.
After each push, the code is sent to the travis.ci system , where the prescribed tests are run, and the code is deployed to NodeJitsu and Heroku.
And if something does not work as it should, a letter arrives immediately, and the icon in the repository changes its color to red.
If on some of the services Cloud Commander is not responding, on the site , at the very top, not green circles, but red ones are displayed near the links. If the answer is long - yellow.
The project uses Gulp , which automates all routine actions: checks js, css, runs tests, etc.
Once I came across an article that talked about the style of naming commits adopted in Angular. This is actually a very important process. Changes, fixes, refactoring, etc. have their own console, and during commits release with prefixes
I want to thank the reader for having come this far (even if he simply squandered). I hope the article was useful and interesting. Perhaps there will be a continuation, wait and see.
This is my first article on the hub, if there are typos, suggestions, comments - I ask in PM or in the hidden branch in the repository . I will try to correct myself.
Website: cloudcmd.io
Demo: heroku , jitsu
Foreword
The first computer book I read was Windows: Wizard Lab . She talks about a variety of utilities for Windows 9x. Some of them no longer exist (Zip Magic 2000, for example), while others are actively used, and most importantly, are developed to this day (Total Commander). Most of all I liked the section about file managers. I didn’t have a computer yet, but already then I realized that it wasn’t serious to use Explorer, and it’s much more correct and convenient to use Two-Panel File Managers. I tried everything that was in the book at every computer that I managed to visit. Most of all, of course, I liked Total Commander. He is the best in his business, it is indisputable.
A few years later, I got a computer. After some time, next to Windows, I installed Linux, and wanted to find something suitable for convenient file management. I didn’t really succeed. Yes, Midnight Commander in * nix is the best, that's true. But many of the functions that I was so used to using Total, were not in it. They were not in the graphic managers. One of these functions is moving the pointer of the current file while entering the name (when there are a lot of folders, and I have a lot of music - flipping through the list is not the most pleasant of occupations).
In the end, I was more pleasant to listen to music, watch videos, and manage files in a bare console. Which I stopped for a while.
Causes
A few years later, getting a job in a small company, I realized that getting into my computer would be much less likely. And indeed, the circumstances are such that more often I work for other people's computers. And since getting used to the new is not very easy for me, I began to increasingly use languages and development tools that work in the browser and do not require installation, configuration, or other lengthy things. I started using Cloud9 , Koding and, of course, GitHub .
I got excited about the idea of cloud services, I liked the openness and capabilities of these projects so much that I began to make my own. This is the file manager of Cloud Commander.
Analogs
Most analogues are written in php, which is not very close to me. Most of the code I write in JavaScript, Therefore, accordingly, I would like to see more applications written in a language that may not be the best, but the most common, that's for sure.
Web file managers
There are a lot of file managers for the web. But, practically, each of them has several fundamental problems:
- mimic the interface of Windows Explorer (making the manager less convenient than Explorer);
- file operations mainly occur on different pages, which is not interactive at all (although it can be explained by the fact that managers are written mainly in server languages);
- They work extremely slowly (they don’t use local storage and other HTML5-functionality to speed up work, because they are written in server language, and JavaScript is used only for basic things such as ajax, and this is good if the data is sent to json and not pieces of html- code);
Of the good managers, I can cite, for example, the following:
- ElFinder (contains problems 1 and 3 (as well as 2, try downloading the file and you will see a popup that you need to close yourself)).
- Pydio (looks very promising, but instead of bringing the familiar interface to the web, it creates a new paradigm (not that it would be bad, but still).
The applications described above do their job very well. But from the usual file manager, for example, I expect another. And if there is a need to work in a browser - why not make programs of this type more friendly, not only for ordinary users, but also for more advanced ones who spend most of their time in classic two-panel file managers.
Two-pane file managers
I couldn’t find sane two-panel file managers that would work in a browser 2 years ago, when I started writing my implementation, I can’t find it now. Although I regularly monitor this issue.
At first, I had ideas to continue someone’s undertakings, to connect to the project, so to speak. I found an analogue (unfortunately I can’t remember the link) total commander on SourceForge, with a web interface, it was written in PHP, with which I was playing at that time. All of its code was combined into one file, and I didn’t really want to deal with this. In addition, in those days, I began to be actively interested in Node.js, and I wanted to try it on a new project.
Of the interesting things that have appeared since that time, I can only say that Metro Commander. True, there is no chance to feel it, since it works only on win8, and not in the browser, but natively. But the screenshots look impressive. Otherwise, there are no special movements in this direction, at least in the places where I am looking.
Features
And so, what makes Cloud Commander so interesting that it is worth paying attention to? Let us examine these points in more detail since it is really important.
Of the most interesting features it is worth highlighting the following:
- Work in the browser. This is an important feature, which entails a series of strengths and weaknesses, on which everything depends. Probably the main thing is that if you don’t use a browser (do such people really exist?), This program is definitely not for you (in other matters, it’s not so difficult to change what is shown in the next paragraph).
- Client server architecture. This follows from the first paragraph, to do absolutely everything that you want to do with the browser does not work. But this implies a huge plus: separation of responsibilities (different parts of the application are involved in logic and graphics, which allows, for example, writing a native client for any OS without changing the server code), which, in turn, allows you to rewrite part of the application to another in the future language, guided by the client-server communication protocol (this will be discussed more below).
- The code is written entirely in JavaScript, which means that parts of the code are completely reused, both on the client and on the server, which makes you take more seriously what you write. Design the code so that it does not depend on the environment in which it is executed, be it a browser or node.js.
- Ability to work with disabled JavaScript in the browser. Most of the operations will not be available, but walking through the file tree, and downloading the necessary things will be possible without problems.
- Modular structure. From the very beginning it was clear: if you write everything from scratch, then in the end you can come to nothing. Therefore, only the main part (core) is written from scratch. Such functionality as: view, editor, console, etc. These are modules written by other people. Any of them can be replaced (if it ceases to be supported, or if a serious analogue appears), and some (the editor and console) have already been replaced, but more on that later.
- Responsive interface. The file manager will work in the browser of the mobile phone and tablet. If the screen does not fit the second panel, only one will be displayed. And it will respond to wheelbarrows, not clicks.
- Ability to interact with Cloud Services (DropBox, GDrive, etc.)
- Ability to download and upload files using Drag'n'Drop.
- Ability to load data from a file (on the desktop) into the editor using Drag'n'Drop.
- The ability to change jshint rules by editing the .jshintrc file, which will affect all js files loaded into the editor.
- Various optimizations to speed up loading / unloading (localStorage, Diff, zip compression on the client). Since most of the work is data exchange between the client and the server, various measures have been taken to minimize the amount of traffic and, accordingly, increase the speed of information exchange (which is the narrowest link in the chain).
Differences
The main difference is the two-panel interface with the usual keyboard shortcuts. But that's not all. In contrast to other browser file managers, Cloud Commander has a console and editor.
The editor supports syntax highlighting in more than 110 languages. The file format is determined by its extension. The source code for Cloud Commander is corrected in it itself (and this article is written in it too).
The console allows you to execute commands on the server, and it does not matter whether it is Linux, Mac OS or Windows. The server on which Cloud Commander is running is controlled from the console directly in the browser.
An important difference is also Node.js as the platform on which the File Manager runs. No more php + apache.
disadvantages
The presence of advantages also implies the presence of disadvantages, without this in any way. The most significant:
- Without node.js, nothing will work.
- In the absence of a browser, it will be possible to communicate with the server only through HTTP requests.
- With JavaScript disabled, most of the functionality will not work.
These are fundamental flaws. If we talk about pressing issues, that is, such. In the editor:
- There is a problem with tabs, if not 4 spaces are used, namely a tab.
- In opera mini, everything looks very scary, and the old IEs are generally thrown back.
- You cannot upload / download a folder using Drag'n'Drop (or download several files at once).
- There is no integration with cloud services at the server level (in the future this may well change).
- There is no progress bar when copying / moving / deleting files.
- Standard dialog boxes (alert, confirm, prompt) are used.
Composition
As mentioned above, Cloud Commander consists of a client and a server. A little has already been said about the client, so we will probably start from the server.
Server modules
The server can work without the installed dependencies, while it goes into limited operation mode. In this mode, the console does not work, js / css / html optimizations are not performed, copying, moving and deleting folders does not work. After installing the modules prescribed in package.json, the following are used:
- express web server which works much faster than the built-in;
- minify module that minifies js / css / html;
- modules for copying, moving, deleting folders;
Common Modules
Some modules work both on the client and on the server, such is diff-match-patch , developed for a long time, but it works very stable.
Customer Modules
On the client, the list of modules used is much wider. It:
Internal organization
Archiver
Most of the questions are probably caused by the archiver on the client. It is used to reduce the size of data sent to the server by the editor. This mode can be turned on (and off) in the settings. In fact, as I said, the bottleneck in the client-server application is data transfer. Packing (and unpacking on the server), in turn, is extremely fast.
Diff
But the text does not have to always be sent entirely. It is advantageous to send to the server only the data that has changed, that is, the patch. Therefore, if, between edits, the file on the server did not change - a patch is sent and applied on the server (provided that the file does not weigh very much, because each such operation is to load the entire file into RAM), if changed, the compressed data is sent . Due to this, the data processing speed is very close to the desktop application.
Local storage
Downloading data from the server is also not needed every time you open files. Therefore, when opened (files) are placed in localStorage, in a place with a sha-1 hash. And, if the hash has changed (without our knowledge), the file is downloaded again, otherwise the hash is updated every time the file is saved. The same is with directories. If the option is enabled, the contents of the directory are loaded once, and to update it, press Ctrl + R (or delete / create a new file / folder).
Advanced Module Loading
In such a large application with a huge number of modules, it would be very difficult to figure out if all the files were loaded immediately, when the page loaded. Such an application would be difficult to maintain, as some programs are updated from time to time, while others become obsolete and require replacement. Therefore, the application is divided into modules: client and server.
Client modules are loaded as needed. If a person needs a menu, it loads, if the console didn’t come in handy for the entire session, its files did not even load and did not start executing, which significantly increases the speed of work and saves resources both on the client and on the server.
Join
Threads node.js is a very powerful tool that is fundamentally different from what other scripting languages have. In the process of immersion in node.js, the idea that you can combine files into a stream and give it as if the file is one, did not leave me. I thought there would be delays in speed, but no. Everything works like a clock, and there is no particular slowdown, but instead, it becomes possible not to combine files into one, and not load them sequentially, but to load them as one file .
This idea, recently, began to promote in jsDelivr . And, it seems to me, this is the right direction.
In short: if you need to download the jquery.js and jquery.fancybox.js file, you can do it this way:
cloudcmd.jit.su/join/lib/jquery.js:lib/fancybox.jsUsing the “:” symbol, file names are separated from each other, so you can combine absolutely anything, and this should not particularly affect the speed of the server, since the files are read sequentially, but immediately after reading they are given to the client.
If the file cannot be read, an error message is sent instead, after which the stream closes.
Development
Enough has been said about the application itself, but there are a few things that I would like to say about development. As already mentioned, Cloud Commander is written in itself.
The project is hosted on a github . It has two branches: dev and master .
In the first one there is an option that is being developed, all the changes are made there, and although ideally everything should always work, sometimes different incidents happen.
In the second branch is the latest stable version. It can always be taken from the repository, everything should be fine with it.
Continuous Integration and Testing
After each push, the code is sent to the travis.ci system , where the prescribed tests are run, and the code is deployed to NodeJitsu and Heroku.
And if something does not work as it should, a letter arrives immediately, and the icon in the repository changes its color to red.
If on some of the services Cloud Commander is not responding, on the site , at the very top, not green circles, but red ones are displayed near the links. If the answer is long - yellow.
Task runner
The project uses Gulp , which automates all routine actions: checks js, css, runs tests, etc.
Commits
Once I came across an article that talked about the style of naming commits adopted in Angular. This is actually a very important process. Changes, fixes, refactoring, etc. have their own console, and during commits release with prefixes
featureand fixvystaskivayutsya of stories, and displayed in a certain form in ChangeLog, it's all done with a single command: gulp changelog.Afterword
I want to thank the reader for having come this far (even if he simply squandered). I hope the article was useful and interesting. Perhaps there will be a continuation, wait and see.
This is my first article on the hub, if there are typos, suggestions, comments - I ask in PM or in the hidden branch in the repository . I will try to correct myself.