RealSync - one-way real-time source synchronization for web developers

Introducing the public utility RealSync (GPL). Her mission is to facilitate the work of those who periodically suffer from the lags of the Samba network folder when searching / editing web project files. The idea of ​​RealSync is that you now work with site files on the local machine in the familiar IDE, and look at the result, as before, on the remote development web server, where RealSync copies the changes in real time. As a result, you can, for example, run a search on all files in the IDE - they are local and not connected via a network folder via Samba, so the search works very quickly; at the same time, your Ctrl + S continues to get to the server instantly, as when working through a network folder.

Realsync- a utility for Windows, MacOS and Linux that allows you to contain in real time on a remote server an exact copy of files (for example, scripts in PHP, Python, Ruby, etc.) from a folder on your local computer, even in poor communication conditions when you work from home. All changes made in the local folder reach the server almost instantly (about 0.2 s delay), regardless of how many of these changes and how they were made (at least through the IDE, at least through Notepad or Far).

The main difference between RealSync and its analogues is that it is extremely resistant to the instability of the Internet connection, reconnects and timeouts. In this case, an SSH connection is used, access through which is automatically configured at the first start of the utility (i.e., you do not need to mess with the keys - the setting is done interactively).

In fact, it’s almost impossible to “kill” RealSync by accident. You can keep it constantly minimized in the tray and forget about its existence (the CPU hardly eats it). If the utility sees that the connection has been broken for a long time, the familiar RSYNC algorithm is automatically launched to quickly copy a large number of differences. In real time mode, its own protocol is applied over SSH, so that when you press Ctrl + S in the editor, you will immediately see the changes on the server. The file transfer is accompanied by a pleasant “tinkling” (which can be disabled if necessary in the config), and a temporary loss of connection - by reddening the icon (when the connection is restored, the icon will turn gray again, and RealSync will “catch up” with the accumulated changes).

And why this bike when there is Samba or Denver or XAMPP?

Generally speaking, there are several ways to develop web scripts. The first way is to use a local web server. This method has a lot of advantages (more control, improved portability and cross-platform, etc.), and a lot of disadvantages, such as: the potential difference between the local server configuration and the configuration of dev and production zones, the need to either monitor the local synchronism SQL databases, or wait until access to a single maiden database on the Internet is slowed down, etc. We will not consider this method in this article, although it certainly has the right to life (and at least 1 million registered users of the same Denver apply).

The second way is to use a web server installed in the office of the company (or on a test machine in the data center), serviced by regular administrators. Such a web server can have several directories, one for each developer. Each programmer works in his own directory so as not to disturb others, and looks at the result on his subdomain (or on his port number). Here we will consider this method.

Using a remote folder for web development

After all, you need to somehow “access” the directory on the remote server. In the most common case, it is simply connected as a network folder to the local machine via Samba. This method has several disadvantages at once:

1. For a tolerable speed of work, you must be on the same local network as the server.
2. But even in this case, it will be quite difficult for you to start searching all the files of a large project in the IDE - there is too much delay.
3. In the event of a connection break (or, God forbid, you work from home via the Internet), be prepared for strange effects and unpleasant lags.

^{I know many people who, bypassing inconvenience (2), go to the server via SSH and use grep in the console to search through files, and they themselves work in the IDE. It seems to me the height of inconvenience. Moreover, in phpStorm, for example, searching all local files of even a very large project works significantly faster than grep and takes 1-2 seconds (how they did it is a mystery to me; apparently, they index something in advance). Personally, I constantly and comfortably use phpStorm to search for local files - an average of 50 times a day.}

There are, of course, those who like to work in VIM in the SSH console directly on the server; I respect them, but I’m unlikely to join their camp: after all, the conveniences provided by a good graphic IDE outweigh for me.

By and large, RealSync is a solution to the IDE speed problem.

So, to solve the problem of search speed, you need to store the source on a local disk and search already on them. And immediately the question arises, how will the source code get to the remote web server when you press Ctrl + S. There are several possible options.

1. Almost all IDEs support transferring a modified file (via FTP or SSH) to a remote server by pressing Ctrl + S. But there are only two problems. Firstly, you cannot modify files “by” the IDE (for example, make git pull from the command line on the local machine). Secondly, if you disappear from the reality of the Internet for a few days and then return with the changed files, the IDE will no longer be able to determine which files have changed (especially if they were accidentally changed on the server for some reason, independently of you), and at best, it will start a full copy, which takes half an hour.
2. Run in the perpetual cycle of RSYNC. This more or less works when the project is not very large, when the Internet is good and when you are NOT on Windows (in Windows, RSYNC is very slow at startup). In all other cases, be prepared for 2-3 second delays.
3. Use a special synchronization utility such as RealSync, Unison, WinSCP, etc.

RealSync was conceived as a tool that provides stable operation in conditions of poor communication and does not require attention from the programmer ("run and forget"). This is how it turned out, unlike numerous analogues.

And if I need two-way synchronization?

This is a very popular question related to the fact that RealSync always overwrites any changes made directly to the folder on the server (if not immediately, then at the next reconnect for sure). Only changes in the local folder make sense and priority.

But let's think about why two-way synchronization might be needed.

• Do you want to work, for example, with Git in the console on the server? But why? After all, you can put Git on your machine and enjoy all the benefits of local work and different GUIs for Git.
• Do your scripts write to some local folder within the document directory? Add this folder to RealSync exceptions (this is done easily, with one line in the config or at the very first start). In addition, making the document directory writable by scripts is an antipattern.
• Do you work on a laptop at home and on another desktop - at work and want to sync files through a web server folder? But a web server is not a means of synchronization and version control. Use Dropbox better or commit to version control system.
• Are you actively using symlinks in a working copy on the server? Well ... but did not try to stop using them, will it at least simplify support and deployment in the future?

In general, you need to understand that RealSync is not a synchronizer, it is a replicator . If you stop perceiving the folder on the web server as something independent, and consider it simply an automatic “mirror” and stop going into it with your hands on SSH (really, why do this?), Everything falls into place. And in most cases, it turns out that two-way synchronization is simply not needed (but if you still think differently, use Unison - if it takes root, of course.)

RealSync installation and first launch

Everything is quite simple: first you need to download RealSync and copy it somewhere, and then somewhere (in Startup or on the Desktop) place a shortcut with the following command line:

"C: / Program Files / dklab_realsync / realsync.exe" Full Path To Your Folders Source
    - or for MacOS and Linux -
perl / opt / dklab_realsync / realsync Full Path To Your Folders Sources

At the first start, you will get something like this window in which you will need to interactively enter the server address, your SSH username and password on it (the password is requested exclusively by the SSH process and is not stored on the local machine, see below), the name of the folder on the remote server etc.

_{If you already have password-free access to the server using an SSH key, the password will not be requested at all, even for the first time. If you do not already have one, RealSync will generate the public + private key itself, the open one will rely on the server, and the private one will be generated in your home directory on the local machine (in a special folder that RealSync perceives).}

The rest of the configuration settings will be written to the file FullPathToYourYourFolderSource / .realsync, so the next time you start, no questions will be asked , but the synchronization will just start and the program will minimize to tray (for Windows).

Now you can edit any source and make sure that they instantly get to the server. If you click on the icon of the launched RealSync in the tray, then the work process can be observed approximately in this window:

^{The RealSync distribution for Windows is self-contained: it already includes RSYNC, SSH, a mini version of Perl and other necessary utilities, so you just need to run realsync.exe and not think about it. On MacOS and Linux, all these utilities are usually already available on the system, which is why they are used.}

So, RealSync hangs in the tray, you never turn it off, so you have the complete illusion that you are working on a local machine with a remote folder. At the same time, you have all the benefits of local search and working with the IDE. That was the purpose of RealSync development .

I will be glad if the utility makes someone else's daily work easier. Comments are welcome. GitHub-e project: dklab_realsync .