Virtual creatures and their habitats: the past and present of TTY in Linux

Ubuntu is integrated into Windows 10 Redstone, Visual Studio 2017 has acquired development support for Linux - even Microsoft is losing ground in favor of a growing number of Torvalds supporters, but you still don’t know the secrets of the virtual terminal in modern distributions?

Want to fix this gap and open the source code? TTY, MASTER, SLAVE, N_TTY, VT, PTS, PTMX ... Pile of concepts, virtual devices and messy magic? All this adds up to a pretty logical picture, if you remember how it all began ...

1. START FROM SCRATCH & KEEP CALM

TTY: PALEOZOI

We stepped straight into the thirties of the 20th century and ended up in the very young Teletype Corporation. Right in front of us is the One-From-Whose-Everything-Begins - a teletype, which is a "direct-printing telegraph", which transmits text messages between two subscribers.

Subscriber A types on the keyboard characters that are converted into electrical signals. On the most ordinary cable, the signals “run” to the teletype of subscriber B and are already printed there on ordinary paper itself. If the signal is duplex, then we are very lucky, and subscriber B can immediately write his answer; if not, he will first need to connect a second wire for feedback.


Here, at Teletype Corporation, they still don’t know what the future will be for their product in the near future, and certainly they don’t suspect that the TTY abbreviation will outlive the teletype. Let's not spoil the intrigue for them, let's move on.

TTY: MEZOZOI

Forty years have passed, we are in the laboratory of Digital Equipment Corporation, we admire the first mini-computer (interactive!) PDP-1. For the input and output of information, as well as to ensure interaction with the user, a teletype already familiar to us is connected to it.

The fact is that leading engineering minds decided not to reinvent the wheel and adapt the existing cheap and affordable mechanism to new needs. A teletype was directly connected to a computer (and not to another teletype, as it was before) and called this business a console . The input operator sees how the typed characters are instantly printed on paper, but this happens without the participation of the OS - thanks to the principle of a typewriter being saved in the console .

TTY: PALEOGEN

We find ourselves at the very beginning of the 80s, this time at Bell Laboratories. One of the most important releases of the “early” UNIX, Version 7 for PDP-11, has just been released here. The features of this release are as follows: the command entered by the user is now displayed according to the ECHO principle (the character typed on the keyboard first goes to the accumulation buffer and only then the OS sends instructions to print this character), simple editing options for the input commands are supported (you can "erase" the character or whole line, move the carriage), the separation of modes appears:

• raw mode (line editing is not performed; escape sequences are recognized as ordinary characters; the entered character is immediately transferred to the process);
• cooked mode (special characters are recognized and stop and interrupt signals are generated for the process; the finished line is transferred to the process only after pressing the Return key).

The quite expected question: how can one "erase" what a teletype has already printed? For visual editing operations, Unix Version 7 provides the printing of certain characters: for example, @ - erase the entire line, # - erase the last character. That is, if our teletype printed ld @ lk # s, and the operator pressed Return, then the ls command went for execution. This is not TTY LINE DISCIPLINE (we will discuss it later), but it is already a big step forward in terms of processing input at the OS level.

By the way, Digital Equipment Corporation for these 20 years not only developed the mentioned PDP-11, but also thought about how to improve teletype: the so-called smart terminals appeared.

We look to the right: this is VT100, one of the first terminals that can work in love and harmony with PDP-11 and is supported by Unix Version 7.

At the OS level, both the console and the smart terminal are now perceived as character devices that connect via the UART interface, which converts the asynchronous data stream into a sequence of characters. For the OS, they are basically identical, the only difference is whether to erase the characters on the terminal screen or print the face symbol using a teletype.


On the left, a general diagram of the interaction of a computer and a console (or smart terminal) is shown. This scheme has one drawback that does not please the operator of the PDP-11 console at all: one session (or session) is associated with one console, in which the user can start several processes in the background, however, only one TTY will be active at one time one.

And our poor operator is forced, in the literal sense of the word, to switch from one console to another, if it suddenly occurs to him to work with several sessions.

TTY: NEOGEN

We found ourselves in the editorial office of the PC MAGAZINE magazine; we are considering the latest issue of January 13, 1987. One of the turns actively convinces us not to spare money on a PC with UNIX System V. What are the arguments? In particular - grep, awk, sort, split, cut, paste, vi, ed - word processing has clearly stepped forward. And the most interesting: terminal emulators are now at our service! Thanks to virtual consoles, it is already possible to launch as many as four sessions without the need to connect more and more physical teletypes.

In addition, the life of monochrome terminals can now be flourished: CGA, Hercules, EGA graphics are supported. A poor operator can breathe easy, a teletype (as well as a smart terminal) in the form of an iron beast threatens him only in nightmares.

We advise the operator only one thing: do not go into the / dev directory for anything - after all, several ttyX await him there and remind you that the same good old teletype lives under the hood of the virtual console.

TTY: ANTHROPOGEN

At the previous step, we made sure: we made the console virtual (the solid edition of “PC MAGAZINE” will not lie). What does it mean - the entire I / O mechanism has been rewritten and fundamentally changed? Then why is the virtual device still ttyX? It's simple: the virtual console is emulated as the most physical one, and the place of the UART driver is probably taken by someone else. We will discuss the changed scheme in detail a little further.

There is one step to the finish line, but we cannot miss it, at least out of respect for Linus Torvalds. Now is 1993, and we finally have the good fortune to examine the source code for Linux 0.95. Why exactly this release? It is in it that the TTY abstraction has already formed, which is closest to what we have in the most recent distributions: three separate layers took shape (TTYX - TTY_LINE_DISCIPLINE - TTY_DRIVER ).

In addition, Linux 1.0 will be released just a year later, where the window interface provided by the XFree86 project will appear. From this moment on , virtual terminals will also be added to the virtual consoles , which the user (in almost unlimited quantities) will be able to launch without leaving the graphical shell ... However, before plunging into the subtleties and details of the improved io-magic, we will return to our present to Ubuntu 04/16.

2. STOP BEAT AROUND THE BUSH & LOOK INSIDE

VIRTUAL CREATURES AND THEIR PLACES

Only some devices in the / dev / directory are used daily: / dev / sdaX, / dev / mem, / dev / zero, / dev / random ... But there are several groups of devices that do not often attract our attention, but more than deserve it. These devices are / ttyX, / vcsX, / vcsaX, as well as / ptmx and / pts / X. As a matter of fact, they will be discussed further.



And our first object is a virtual console. Each such object has at least a sacred identifier and a totem animal file of the virtual device / tty, of which there are already 64 in the virtual forest of the / dev directory. We

’ll check if we can communicate with them. We execute Ctrl-Alt-FX (or chvt X, where X is the console number, for example, Ctrl-Alt-F1) and notice that X can be equal to 1, 2 ... 6. At the same time, the virtual console opens before us, at the first start we are prompted to enter a username and password and create a new session for us. If X is 7, then we return to our native graphics penates and understand that / tty7 is associated with XServer. Move on. Eight, nine, ten, and so on to 63 — no signs of life.

The fact is that on Linux there is a macro MAX_NR_CONSOLES (64) that determines the maximum allowable number of virtual consoles, which are represented by 64 virtual device files / dev / ttyX. However, the last word remains with the ACTIVE_CONSOLES parameter (/ etc / default / console-setup), and this parameter defaults to six.

The consoles are initialized in several stages. First, the kernel, having received control from Grub, during the initialization of the subsystems, calls the console_init function, which creates the primary console, the boot console, designed to display debugging information. This console performs the output of characters in the most primitive way: through "putchar", which directly accesses the BIOS, initializing and populating the biosregs structure, and outputs the character to the console using the 0x10 interrupt.

Later, during the execution of “fs-initcall” and “console-initcall”, virtual devices and structures are created for 6 full-fledged virtual consoles - “real console”. The activation of these consoles is performed by the first kernel process / sbin / init, which launches the getty program, which reads the configuration files /etc/init/console.conf and /etc/init/ttyX.conf and subsequently displays the contents of the welcome file etc / to the console issue and launches login. Next, XServer initiates the activation of the console on dev / tty7, on which the graphical shell runs.



However, we still have questions. What is the unknown object / dev / tty0? And if every / dev / ttyX is a virtual console device, then why do we need / dev / console and / dev / tty? For the answer, go to tty1 (by pressing Ctrl-Alt-F1) and run the following script in the background:

sleep 10
echo “tty0” > /dev/tty0
echo “tty” > /dev/tty
echo “console” > /dev/console

Then go to, say, tty4 and wait a few seconds. After the end we see the following picture:


The distribution of roles becomes clear: / dev / tty0 = / dev / console = current console, i.e. both are always associated with the console that we currently see in front of us, and / dev / tty “remembers” the console with which the process started. Therefore, while our process was running, / dev / tty0 and / dev / console were determined for it in the course of the play, depending on the current active console, but / dev / tty remained unchanged.

We are in no hurry to leave the / dev directory. There are even a little more than a dozen interesting objects: / dev / vcsX (virtual console screen) and / dev / vcsaX (virtual console screen with attributes). Another experience: we move to tty5 and leave some traces of our stay, then go to any other console (let it be number 3), make “cat” on / dev / vcs5 and see exactly the state of console 5 in which we left her a few seconds ago. In this case, respectively, / dev / vcs3 and / dev / vcs (as well as / dev / vcsa) belong to console 3, which we are currently on.

We understand that / dev / vcsX is nothing more than a memory poola console virtual memory device that allows us to seamlessly move between tty instances. Paired with it is / dev / vcsaX, which provides basic information about the state of the screen: colors, various attributes (e.g. flicker), current cursor position, screen configuration (number of rows and columns). To summarize what was seen by the scheme:

TOP VIEW BETTER

Now let's stop with the study of tty zoology for a while and move on to the tty abstraction itself, of which our virtual devices are a part. Let's look at the general structure of the tty-complex and select three components:

1. / dev / ttyX is the virtual console device in the file system that has taken the place of the UART driver and with which we are already familiar. The devices / dev / vcsX and / dev / vcsaX are located at the same level, communication with them is carried out directly through / dev / ttyX.
2. TTY Line Discipline is a driver that makes ECHO a typed command and gives us the ability to edit it. Also, the driver of this layer generates signals during a set of control sequences (^ C, ^ Z, etc.). By default, N_TTY reigns here, however, this module can be replaced, for example, with your driver - we'll experiment with this a bit later;
3. TTY driver is a driver that provides a set of methods for initializing and opening the console, as well as methods that process input / output operations, pausing the console when switching and resuming its operation, and, of course, ensures that the command received from the user is transferred to the active process.

Remember the complaints of the PDP-11 operator? He didn’t like spending time moving from one physical console to another. Now the situation is as follows: we have by default 7 virtual consoles, and in front of them is an office chair on wheels (of course, also virtual). When we switch from one console to another, the operating system moves our chair to the desired tty, and together with the chair, a set of physical io devices also “switches” to it: the monitor now has the state of our new console, it also receives keyboard input and etc.

At the same time, the processes from the first tty continue to work: they read commands from the file of their virtual console, write to this file, but - since they are torn from the "chair" - they do not receive any events (the same ^ C and ^ Z) and - since physical devices “left” with the “chair” - they can only accumulate their “output” in the buffer to send it to the monitor as soon as the “chair” returns.

JOURNEY TO THE EARTH CENTER

Yes, everything looks quite presentable from above. But you,% username%, probably want to see how the three-level interaction of tty-components is implemented directly in the code? For the answer you will have to go down from heaven to earth, even better to say - underground, into the bowels of the Ubuntu source code (we will work with the kernel version 4.4).

Let's make a preemptive move - let’s figure out through which structures the tty-abstraction is linked into a single whole.

Firstly, it is tty_struct, which has a tty_ldisc field (this is the structure of the 2nd layer driver methods), a tty_driver field (this is the 3rd layer driver) and then tty_operations (this is the structure of the driver methods 3rd layer, for the sake of convenience, rendered directly in "tty_struct").


That is, tty_struct provides access to the TTY_LINE_DISCIPLINE and TTY_DRIVER layers. Got access to it - 2/3 of the stack of tty-abstraction, consider it in front of us. Now we need to understand how the transition from virtual device files to this structure is carried out. The answer is simple: the structure "tty_file_private" just has a field of type "tty_struct". Therefore, referring to the file of the virtual device at the 1st level, we easily get access to higher levels.

While the puzzle is developing, but this is not enough for us. We’ll go through the kernel (using qemu and cgdb) and consider the backtrace of the output (echo) of a single character entered by the user from the keyboard:


So, we are at tty-stack I level. There is a system call “write”, which on our tty is processed by the function “tty_write". A pointer to the file structure of the virtual device and a buffer with a symbol are passed to it. In the “tty_write” function, an instance of “tty_struct” is obtained from the file. Accepting the game, "tty_struct" first calls the driver TTY_LINE_DISCIPLINE - "tty_ldisc", which defaults to N_TTY. The first level is completed!

N_TTY takes the baton: in turn, it calls the n_tty_write method, and then passes the buffer to the output_process_block function, which, making sure that we have entered a non-carriage position character, asks tty_struct to call tty_driver. That's right, we are moving to level III.

"Tty_struct" successfully plays the role of an intermediary, and now - the "con_write" method of the tty-driver named "console_driver" is already running. A third-level driver would be happy to do its job, but it is one, there are many consoles - which one should I work with? “Tty_struct” again comes to the rescue and gives the driver the required instance of the “vc” structure (it is responsible for the state of its particular console and contains its keyboard, screen settings, as well as a set of graphical display methods).

Console_driver locks the console and invokes vc_data to finally echo the character. “Vc_data” realizes with horror: they turned to her not for the question of the well-being of the console entrusted to her, but for the sake of action. This means only one thing: it's time to call for help the “consw” methods that VGA in our case represents (with a different kernel configuration, it could be, for example, framebuffer). And for sure - VGA gets down to business, hides the cursor, prints a character, scrolls the screen if necessary or moves to a new line, moves and displays the cursor. “Vc” exhales: “console_driver” accepted the job and the console unlocked. We can exhale, too, because all three levels have been successfully completed, each component has fulfilled its mission.

3. KEEP AN EYE ON VIRTUAL TERMINAL

But that's not all: it's time to get acquainted with the representatives of another class of inhabitants / dev - with terminal emulators . These are the same xterm or gnome-terminal that we launch from the console equipped with a graphical shell, using, for example, Ctrl-Alt-T or Ctrl-Shift-T.

They live in a separate aviary / dev / pts (= pseudo-terminal slave) and are files numbered 0, 1, 2, etc. We execute ps in the current terminal and see - we are on / dev / pts / 1. Press Alt + 5 - we move to our fourth in order of opening terminal, whose virtual device file is / dev / pts / 20. At any terminal, bash meets us, each terminal has its own set of processes. So far no surprises.



But note: / dev / pts / X are created dynamically, run on the same console / dev / tty7, do not require login and the most interesting thing is that there is no “chair” rule: we can open several virtual terminals and simultaneously observe how the work on each of them. Once again,% username% may have doubts: is the principle of tty-abstraction preserved here and how does the slave device fit into this principle, which is probably controlled by a certain device - master?

The new object is not long in coming: in a single copy, the ptmx file is in the same directory / dev (Actually, it may not be the same: if more than one console with a graphical shell is running). According to man, opening / dev / ptmx creates a subordinate part of the pseudo-terminal / dev / pts / X associated with its leading part “ptm” (access occurs through the file descriptor, but the real file is not created). Then “ptm” is passed to the grantpt and unlockpt functions, and after all this, you can directly open / dev / pts / X, which will behave exactly like a virtual console (except for the features described above).

For us, in the spirit of the idea of ​​tty abstraction, this means the following: when a user wants to run a terminal emulatorXServer calls on / dev / ptmx to create the virtual device / dev / pts / X. The powerful “multiplexer” / dev / ptmx kindly does this, assigns the device file to the terminal instance and ... / dev / pts / X takes the place of / dev / ttyX, the layer driver TTY_LINE_DISCIPLINE is assigned to it, and TTY_DRIVER gently takes it into its arms. The stack over / dev / pts / X takes on the familiar look. The task of studying the mechanism of the terminal emulator smoothly boils down to the previous story with the virtual console , but its detailed study requires a separate article (which is included in the plans for the future!).

4. LET'S PLAY WITH TTY LINE DISCIPLINE

For a second, remember tty- “Paleozoic”: there was a time when the TTY_LINE_DISCIPLINE layer was not even a separate layer and did not have full-fledged modern functionality. Let's try to estimate the weight of the changes that have taken place since then.

First, make sure that we are really dealing with N_TTY:


Everything is known in comparison, therefore we act radically and with the help of stty we disable all useful N_TTY features:


The result clearly demonstrates the area of ​​responsibility of N_TTY, without which the output is not formatted, the input is not displayed. Moreover, if we open a new terminal, we will see the integrity and integrity of its LINE_DISCIPLINE. The effect we get suggests that we know exactly which component processes all of our input, we can modify it for each virtual terminal individually, and, I remember, we heard that this component can be replaced by loading our module.

Unfortunately, N_TTY is itself part of the kernel. Therefore, we will take other drivers of the LINE_DISCIPLINE layer, provided in Linux and loaded as modules, as a basis. In their image and likeness, we modify the source file n_tty.c:

1. Add the module shipment function, in which the tty_register_ldisc function is called, which makes the kernel “familiar” with our personal discipline line. The first parameter to this function is its unique identifier, and the second is a pointer to a structure with driver methods.
2. Add the function of "unloading" the module, in which the function tty_unregister_ldisc is called, respectively.
3. In the structure "tty_ldisc_ops" we will set a new driver name.
4. We will make sure that our module “recognizes” the functions it needs from the tty_io.c file (it does not please us with the “EXPORT SYMBOL” macros, which forces us to either add all the required functions manually or link with tty_io.c).

Now add some functionality that distinguishes our line of discipline from the original. Remember that it is TTY_LINE_DISCIPLINE that processes the service sequences, so it’s a sin not to conjure in this field. To do this, open the function "n_tty_receive_char_special", in which TTY_LINE_DISCIPLINE checks whether the entered characters are special and when they are found sends a corresponding signal. For example, swap the signals generated for Ctrl + Z and Ctrl + C:


After that, we will get our_ldisc.ko kernel module directly from our modified file. Download it, make sure that the download was successful. Check that "our_modyfied_ldisc" is indeed registered as TTY_LINE_DISCIPLINE. Let's open the terminal and see the pts number. After that, we will make our driver responsible for the TTY_LINE_DISCIPLINE layer at / dev / pts / X:


We will set up a new discipline line using the “stty echo cooked” command - now the terminal works in the usual mode for us. Run the test program with an eternal cycle and compare the effect of Ctrl + Z and Ctrl + C:


We have achieved the desired result: signal generation has been redefined at the level of the layer driver TTY_LINE_DISCIPLINE individually for one terminal emulator! There is a field for the work of fantasy: from tricks with processing service sequences to a custom filter of commands.

FINALLY

Now for you,% username%, the secrets of virtual consoles and terminal emulators are no longer secrets, promiscuous magic is not magic, but technology that has come a long way to create a flexible tty subsystem, and teletype is not an artifact of antiquity, but an invention (by the way , ours, domestic), without whose descendants I don’t feel like presenting a modern computer.

We love to tell fascinating stories. Do you want to listen to them live? Come to the NeoQUEST-2017 Face-to-Face , there are many interesting reports waiting for you : from hardware to cryptography! Admission is free when registering on the site.