Terminal access to DBMS Caché - now also in the browser

With the development of web technologies, more and more useful services, applications, programs and even games appear in the browser window. The time has come for the Caché DBMS terminal .

Under the cut you will find a description of all the charms of the application and the history of its development.

Functionality

The following is valid for a web terminal:

• Executing arbitrary code and Caché Object Script commands, terminal utilities, and programs
• Convenient SQL mode for quick access to the database
• Autocompletion of Caché Object Script keywords: classes and their methods, properties, parameters, globals and variables in the current namespace
• Monitoring changes in globals and files (like tail -f)
• Team History
• Caché Object Script syntax highlighting
• Definition of abbreviations
• Multiline editing
• Customize application behavior and themes

All of the above works on any Caché server where WebSockets support is present. Just follow the link to the web application and get started.

You can find a detailed description of all the features on the project page .

Security

Naturally, the main goal of any web application should be its security. Since communication with the server occurs through an open port, the latter needs protection. The first thing that a terminal port will require you to enter when connecting is a key of arbitrary length. The server, receiving the wrong key, immediately disconnects. Now a GUID is used, which is generated every time when accessing the main CSP page or in case of unsuccessful connection to the open port. That is, every time there is an attempt to connect to the socket with the wrong key, a new one will be generated, and so on and on, which makes it impossible to select it. The same thing happens when the CSP page is called - only now the key is given to the client and used to establish a connection to the socket.

Simply put, it remains only to establish authorization on the CSP page, thereby preventing the unwanted visitors of the open port from receiving the key.

History of software implementation

If you look at the terminal from the side, it may seem to someone that its mechanics are quite simple. At first, it seemed to me that way too. Understanding the behavior of such applications, along with getting to know Caché, there were a lot of interesting points and difficulties, which will be discussed later. The main task remained to make the terminal familiar for geeks and at the same time friendly for ordinary users, providing new opportunities and finally embellishing the black-and-white graphical interface of a regular terminal, because the window is already 2013!

Further, I will mainly describe the crackdown on the rake that I had to stumble upon and the very ones that all programmers who were familiar with the web or Caché were like. Maybe the methods and solutions presented below can be made even more interesting, and if you know how, it will be very interesting to hear.

Starting to create a new software product, you need to think about how all this should work. Since I was just getting acquainted with Caché, at first I was visited with thoughts that the terminal is functioning quite simply - you just need to execute commands on the server and read the answer from it. I did not worry at all about the front end, since I had to work with it before. But as soon as I began to delve deeper into development, it turned out that it was necessary not only to execute commands, but also all sorts of terminal utilities, configure reading information from the client (for example, when processing the read command), achieve streaming input / output and solve a number of other small tasks.

After a little reflection with colleagues, it became absolutely clear that the WebSocket protocol will be used for data transfer - a relatively new, stable and reliable way to transfer data via the Web. And on the client side, of course, all the delights of HTML5. These are both CSS animations and pure transparent JavaScript.

First of all, sitting down on the client side, I did not want to use ready-made solutions or apply any frameworks, since the terminal is ideally a lightweight application that does not need advanced access to the DOM like JQuery and various magic JavaScript animations. No, in fact, animations might be useful, but in the format of just a bunch of CSS3 properties, no more. The application promised to come out easy, both for the browser and for the user.

I didn’t bother with the display of information for a long time - it is a monospaced font, easy block layout and familiar styles. But I had to think about the field for entering information: it was necessary to implement syntax highlighting. Many plugin solutions were dropped, so the last option seemed to be a concise editable div in HTML5. But there were some charms there - the content had to be translated from HTML to plaintext and vice versa, set and get the caret position, copy the original, unadorned text to the buffer - these tasks are far from being completed in a few lines. In addition, you also need to insert your flashing terminal carriage into the text and implement the usual system Ctrl-C, Ctrl-V, Right click + Paste, Insert, ...

The implementation option for “backlight and carriage” in the input field was very tricky and simple. In fact, we only need to highlight the text, well, and insert the carriage. Suppose we have text that is plain and highlighted. The first is contained in the most typical textarea, and the highlighted one is in a block of exactly the same size. Catch it? Just like that, using several CSS tricks, we make the input field transparent with a transparent font, under which we place a block with highlighted text. As a result, we get a visible selection of the text and complete freedom to edit it. Here, only Internet Explorer, as always, distinguished itself - the carriage in it still remains visible when it is transparent in other browsers. Therefore, I had to abandon the usual flashing terminal carriage in it - let it stay with my native.

An interesting point also arose with the processing of keystrokes - it was necessary to highlight the input data, which means that by clicking the process the contents of the input field. Yes, but you won’t be able to get the contents of this field exactly at the moment of pressing (it’s more accurate, but without the last “pressed” sign) - the DOM does not have time to update before the keydown, keypress events are executed, and by keyup it is not at all interesting to update the visible part of the input, although this also another way out. The second way out is to add the character to the string manually. But the latter immediately disappears in the case of Ctrl + V. Let's make the third method - we will call the handler function of the click 1ms after the click itself. Yes, now we have received input, but the ability to control the event passed to the handler, for example, to prohibit the default action of the key, has disappeared.

Parsing input, syntax highlighting and inserting a carriage into the form was easy to implement - first you need to replace what can spoil HTML formatting, namely the characters "<", ">" and "&" with the corresponding "<", ">" and "&". Then - perform the syntax highlighting by ultra-regular expression (which, in fact, inserts only tags into the text) and only then insert the carriage, determining its “real” position (excluding tags and HTML entities), for which more one method. Yes, all of the above is performed only in this order, otherwise either the carriage itself will be highlighted, or a lot of broken HTML markup will appear.

But it was interesting to work with auto-completion. I copied it three times. And the progress of the algorithm was as follows:

1. Just get a piece of the line from the carriage to the nearest left separator or space and look for matches with the available options in the array of all options.
2. We are looking for the same piece of string, possibly including the characters "$", "%", "##" and others to determine the type of addition in a special object, divided into "categories".
3. Parsing the entire left part of the carriage by the “masks” - the inverse of regular expressions that are contained in the specially structured object of the “terminal dictionary”.

I don’t know if the third method seems to be familiar to anyone, to which I gradually approached, but it was he who showed the most chic and fastest results.

So how does it work? Very simple. All that is needed to create almost any type of autocompletion is competently composed regular expressions in the object of the "dictionary". Here's what it might look like:


Each object inside language can have a special object property “! Autocomplete”. If it is present, the autocomplete parser will pay attention to this object, namely, read its properties reversedRegExp and child.

As you might already have guessed, reversedRegExp is compiled in a special way, and it is he who determines whether it is appropriate to use the properties of the current "dictionary" object (hereinafter simply as a "dictionary") for auto-completion. The storage brackets in the regular expression are used to highlight the part of the search string that will be checked against the dictionary property names (“terms”). This allows you to find the key in any syntactic structure, by which the choice of available options will be made.

With classes, the task is slightly different - you need to get the class name and suggest the properties corresponding to it. This was solved by supplementing the property of the! Autocomplete object with a similar property of the child object, which also contains reversedRegExp - the prefix to the parent regular expression, which will be considered when the latter matches. The verification algorithm is quite simple. If you wonder how exactly this algorithm works, you can find it inside the project repository.

The advantages of this approach are obvious - this is the visual structure of the dictionary of all syntactic constructions, and a rather quick way of auto-completion, which can be expanded in every possible way. Yes, the number used as the meaning of the “term” property is its alleged “frequency of use”. It is on this figure that the proposed options will be sorted.

On the server side, the entire dictionary of autocompletion of classes and methods of the current namespace, in turn, is generated and stored in a JSON file containing an object of objects that, if necessary, will be loaded and merged with the class dictionary object on the client inside the main dictionary object. Here it is.

The server itself was previously taught to send all write directly to the client, using thispieces. But for read, as it turned out, it would not work out with something simple like “+ T”. The whole problem is that when a user tried to execute either a terminal utility, or to compose a script with the reverend read server, processing them in xecut simply hung up or spoiled the input data.

Well, let's say we put the terminal in the processing mode of standard terminators at the input (“+ T”), and we will send them from the client. Well, now read does not freeze, but another situation arises - we are reading the package received from the client, not its body. The package itself contains a bit of “garbage” for us - these are the first few bytes that serve as its header. They had to be discarded somehow.

To make it easier to imagine what you need and how it should all happen, consider the proposed sequence of the “read a write a” command on the server.

• Receiving a command from a client
• Transition of the terminal application to the execution mode - installation of input / output “directly”
• Xecute command transfer
• read a : read data from the client ending in a terminator
• We get the body of a read package and put in a
• the write a : send the client value and
• Completion of execution mode

But how to get this body in the same way so that extra bytes do not get into a (the header of the WebSocket package)? Logically, you need to make some kind of your read handler. Yes, and not simple, but at the system level: terminal utilities also use read.

Fortunately, experienced sql.ru forum guys told me the wonderful undocumented feature of Caché - I / O redirection. With its help, it was possible to do exactly what was needed - to process all arriving / departing data in its own way. Input redirection is done by writing seven subroutines that will take on the functions of the primitive read and write commands when ## class (% Device) .ReDirectIO is turned on. If the detailed implementation of this charms is interesting, this thread may come in handy .

I hope the experience described above will definitely come in handy for someone and will become no less useful than the terminal itself. On the way from the concept to an already functional application, many new ideas have appeared, and this is not the limit - here you can limit yourself only to imagination. Follow or participate in the development of the project on GitHub , propose and discuss ideas, any of your feedback will be useful. Have a nice administration!