March 28, 2016 at 00:02The distributed nature of the Tox messenger
While copyright holders are going to block the centralized Telegram , the community of users of the distributed Tox messenger is growing. Today, according to the statistics of the site www.toxstats.com , Russia ranks second after the United States in terms of the number of users, only 30-50 knots behind.
In this publication, I would like to talk about the distributed nature of this messenger, the general principles of the Tox DHT network, as well as how to " catch up and overtake America " by the number of nodes.
Each Tox node is identified by its IP address, port number, and public 256-bit key. There are two conditional types of nodes:
The public key of the node is used to encrypt packets transmitted to this node. Packets are decrypted on the node side using a 256-bit private key. The UDP protocol is used to transmit DHT packets.
To “enter” the Tox network, it is enough to have connectivity with any Tox node that is already on the network. Usually, a list of known bootstrap nodes on the Internet is used for this . In addition, the libtoxcore library uses sending packets to broadcast addresses, which allows you to connect to the Tox network without having an Internet connection (provided that your network segment has the necessary node). And even without access to the Internet, two or more Tox nodes form an isolated Tox network that allows local clients to interact.
Since each Tox node has a unique identifier in the form of a public key, the relationship between any two nodes can be expressed as an artificial metric based on these keys. This metric is the distance between the keys. The distance is calculated as the modulo 2 addition (bitwise XOR) between the two keys and is interpreted as an unsigned integer.
From the properties of the XOR operation it follows that the zero distance can only be between the same key (node key), and half of the entire key space with respect to the node key will have the maximum possible distance - with some convention, nodes with such keys can be considered "unreachable »And interaction with similar nodes will be a rather rare event.
The process of entering the Tox network begins with sending the " Nodes request " packet to a known bootstrap node (one or more), where the contents of the packet are the sought public key. The node that received the Nodes request package searches for the keys with the shortest distance from the wanted key among the public keys it knows (except for its own and the requested one) and responds with the Nodes response package"containing from 1 to 4 keys found and the corresponding nodes (IP / port). Iteratively repeating Nodes request to the nodes from the client-node response can find another node with the minimum distance from the desired key (while receiving information about the existing intermediate nodes).
If the client node will indicate its own key as the required key, then the found nodes with the minimum distance between the keys will be its “neighbors”. If the client-node will indicate the key of the requested node as the required key, but the found nodes will be neighbors of the requested node, this is how network statistics calculation works.
The vitality of each known node is checked by periodically sending a " Ping request " packet , to which the recipient must respond with a " Ping response " packet . Additionally, with some periodicity, the client node sends a “Nodes Request” packet to a random (known) node to receive information about changes in the DHT network. Non-responding nodes are removed from the list of known nodes after a timeout.
The need to frequently send a “Ping request” packet for a large list of known nodes leads to an unjustified network load. At the same time, saving information only about the nearest neighbors will lead to an increase in the search time for an unknown node. To achieve balance, the concept of a key index is introduced in the libtoxcore libraryIs the index of the first different key bit with respect to the node key. For each new key, its index is calculated and for each index, the kernel stores up to 8 keys, displacing the keys with the largest distance. Additionally, the kernel stores 8 keys of the nearest neighbors with the same extrusion algorithm.
In the current libtoxcore implementation, the index length is limited to 128 “baskets”, which, with some luck, allows each node to store information about 1024 nodes (until some time in the past, while the network was very very small, 32 baskets and 190 nodes were used, respectively). With a minimum packet size of 82 bytes (“Ping request”) and polling each node every 60 seconds, we get traffic of ~ 22Kbit with a maximum load index.
From the rule of calculating the basket index, it also follows that the smaller the distance between the two keys, the larger the index will have a key and the less likely it will be to meet such a key. In the implementation of the libtoxcore library, keys with an index greater than 127 either become "neighbors" or fall into the 128th basket depending on the distance.
Thus, each node of the Tox network maintains effective connectivity not only with its neighbors, but also with nodes at the “far frontiers”, balancing the load on the network and the search time of any other node.
Unlike DHT nodes, all information about which is known or can be obtained by any client of the Tox DHT network, client applications are hidden from an outside observer - just knowing the ToxID of the contact (containing its public key) is not enough to establish the location of this contact. To connect two Tox applications, the onion routing mechanism is used .
The mechanism for establishing communication between two clients can be represented as follows. Two clients (A and Z) announce their public key on the nearest (for their public key) nodes through three random intermediate DHT nodes, each of which knows only its neighbors in the packet routing, but cannot read the contents of the packet.
The second client (Z), who wants to connect to the first (A), also through three random DHT-nodes makes a request to establish a connection to the node closest to the desired key (A), which knows about the announcement of the first client and the route by which to transfer request from the second client.
The first client (A), in case of accepting the connection request, performs the reverse operation to the nearest DHT node of the second client (Z).
After exchanging information about each other's location and temporary keys, clients can connect directly.
If clients do not want to share information about their location even with their contacts, they can use nodes that support TCP-relay through proxies like Tor .
A feature of TCP relays is that they try to use “well-known” (443 (HTTPS) and 3389 (RDP) printers, which makes it difficult to track and identify them.
To protect a distributed network from most known threats, you can use a simple rule of thumb - the more reliable nodes are in the network, the more resistant it is to possible attacks. Moreover, for some types of attacks, the neutralization of each trusted node requires a dozen or two attackers.
Based on the description above, if you use any Tox client, then you are already a network node (while your node is not connected to your ToxID in any way, except that both are on the same host). If you do not use Tox yet, but want to help the project, you can install the bootstrap node on the servers and computers controlled by you - it does not consume a lot of traffic or computing resources (unlike the good old days of Folding @ home ).
A detailed description of the assembly and installation of the node can be found at the link " We join the tox community or install the node in 5 minutes ." However, I tried to simplify this process as much as possible by compiling the tox-easy-bootstrap package for most popular Linux distributions.
To install the tox-easy-bootstrap package, follow the link to the project repositories, select your distribution and follow the simple instructions to add the repository and install the package on your system.
The package installer will automatically receive an up-to-date list of public bootstrap nodes, create a configuration file /etc/tox-bootstrapd.conf and launch the tox-bootstrapd daemonunder a separate system user. Once a week by cron, a special script will update the list of public bootstrap nodes in the configuration file, so you will not need to worry about its relevance in case of a server reboot - “set and forget”.
For cases when a “normally closed” firewall is used on the server, you may need to enter allowing rules for incoming traffic on UDP: 33445 and TCP: 3389,33445 (TCP port: 443 is not used, because the process starts under an unprivileged user) - in order to avoid potential "sabotage", I did not automate this part: Package configuration file /etc/tox-easy-bootstrap.conf
allows you to change the default settings (which are suitable for most cases) and, for example, “link” several of your private nodes in case all public nodes are unavailable - as described above, it’s sufficient to connect to any one node in the network to access the second node.
The question of whether to publish data about your node in the list of public nodes remains at your personal discretion - from a technical point of view, a private node is no different from a public one.
The Tox protocol allows you to not only write bots , exchange messages, files or make audio-video calls, but also use it as a basic protocol for other network tasks.
For example, tuntox and toxvpn projects use the Tox protocol to organize P2P host connections for NAT, adding to the list of Full-Mesh VPN solutions .
There are experimental toxmail projects for organizing a mail message, toxscreen for accessing the remote desktop, toxshare for file sharing between a trusted circle of people.
For experiments on writing new applications, you can use wrappers (supported and not very) for other languages and platforms: python , rust , go , node.js - almost unlimited scope for new ideas and projects.
In this publication, I would like to talk about the distributed nature of this messenger, the general principles of the Tox DHT network, as well as how to " catch up and overtake America " by the number of nodes.
Introduction
Each Tox node is identified by its IP address, port number, and public 256-bit key. There are two conditional types of nodes:
- A bootstrap node is a node that serves the network but does not interact directly with the user.
- Tox client is a node that, in addition to servicing the network operation, performs any additional work (bot, messenger). In this case, the client and the node use different key pairs.
The public key of the node is used to encrypt packets transmitted to this node. Packets are decrypted on the node side using a 256-bit private key. The UDP protocol is used to transmit DHT packets.
To “enter” the Tox network, it is enough to have connectivity with any Tox node that is already on the network. Usually, a list of known bootstrap nodes on the Internet is used for this . In addition, the libtoxcore library uses sending packets to broadcast addresses, which allows you to connect to the Tox network without having an Internet connection (provided that your network segment has the necessary node). And even without access to the Internet, two or more Tox nodes form an isolated Tox network that allows local clients to interact.
Self-organization of a DHT network
Since each Tox node has a unique identifier in the form of a public key, the relationship between any two nodes can be expressed as an artificial metric based on these keys. This metric is the distance between the keys. The distance is calculated as the modulo 2 addition (bitwise XOR) between the two keys and is interpreted as an unsigned integer.
From the properties of the XOR operation it follows that the zero distance can only be between the same key (node key), and half of the entire key space with respect to the node key will have the maximum possible distance - with some convention, nodes with such keys can be considered "unreachable »And interaction with similar nodes will be a rather rare event.
The process of entering the Tox network begins with sending the " Nodes request " packet to a known bootstrap node (one or more), where the contents of the packet are the sought public key. The node that received the Nodes request package searches for the keys with the shortest distance from the wanted key among the public keys it knows (except for its own and the requested one) and responds with the Nodes response package"containing from 1 to 4 keys found and the corresponding nodes (IP / port). Iteratively repeating Nodes request to the nodes from the client-node response can find another node with the minimum distance from the desired key (while receiving information about the existing intermediate nodes).
If the client node will indicate its own key as the required key, then the found nodes with the minimum distance between the keys will be its “neighbors”. If the client-node will indicate the key of the requested node as the required key, but the found nodes will be neighbors of the requested node, this is how network statistics calculation works.
The vitality of each known node is checked by periodically sending a " Ping request " packet , to which the recipient must respond with a " Ping response " packet . Additionally, with some periodicity, the client node sends a “Nodes Request” packet to a random (known) node to receive information about changes in the DHT network. Non-responding nodes are removed from the list of known nodes after a timeout.
The need to frequently send a “Ping request” packet for a large list of known nodes leads to an unjustified network load. At the same time, saving information only about the nearest neighbors will lead to an increase in the search time for an unknown node. To achieve balance, the concept of a key index is introduced in the libtoxcore libraryIs the index of the first different key bit with respect to the node key. For each new key, its index is calculated and for each index, the kernel stores up to 8 keys, displacing the keys with the largest distance. Additionally, the kernel stores 8 keys of the nearest neighbors with the same extrusion algorithm.
In the current libtoxcore implementation, the index length is limited to 128 “baskets”, which, with some luck, allows each node to store information about 1024 nodes (until some time in the past, while the network was very very small, 32 baskets and 190 nodes were used, respectively). With a minimum packet size of 82 bytes (“Ping request”) and polling each node every 60 seconds, we get traffic of ~ 22Kbit with a maximum load index.
From the rule of calculating the basket index, it also follows that the smaller the distance between the two keys, the larger the index will have a key and the less likely it will be to meet such a key. In the implementation of the libtoxcore library, keys with an index greater than 127 either become "neighbors" or fall into the 128th basket depending on the distance.
Thus, each node of the Tox network maintains effective connectivity not only with its neighbors, but also with nodes at the “far frontiers”, balancing the load on the network and the search time of any other node.
DHT Network Clients
Unlike DHT nodes, all information about which is known or can be obtained by any client of the Tox DHT network, client applications are hidden from an outside observer - just knowing the ToxID of the contact (containing its public key) is not enough to establish the location of this contact. To connect two Tox applications, the onion routing mechanism is used .
The mechanism for establishing communication between two clients can be represented as follows. Two clients (A and Z) announce their public key on the nearest (for their public key) nodes through three random intermediate DHT nodes, each of which knows only its neighbors in the packet routing, but cannot read the contents of the packet.
The second client (Z), who wants to connect to the first (A), also through three random DHT-nodes makes a request to establish a connection to the node closest to the desired key (A), which knows about the announcement of the first client and the route by which to transfer request from the second client.
The first client (A), in case of accepting the connection request, performs the reverse operation to the nearest DHT node of the second client (Z).
After exchanging information about each other's location and temporary keys, clients can connect directly.
If clients do not want to share information about their location even with their contacts, they can use nodes that support TCP-relay through proxies like Tor .
A feature of TCP relays is that they try to use “well-known” (443 (HTTPS) and 3389 (RDP) printers, which makes it difficult to track and identify them.
“Catch up and overtake America”
To protect a distributed network from most known threats, you can use a simple rule of thumb - the more reliable nodes are in the network, the more resistant it is to possible attacks. Moreover, for some types of attacks, the neutralization of each trusted node requires a dozen or two attackers.
Based on the description above, if you use any Tox client, then you are already a network node (while your node is not connected to your ToxID in any way, except that both are on the same host). If you do not use Tox yet, but want to help the project, you can install the bootstrap node on the servers and computers controlled by you - it does not consume a lot of traffic or computing resources (unlike the good old days of Folding @ home ).
A detailed description of the assembly and installation of the node can be found at the link " We join the tox community or install the node in 5 minutes ." However, I tried to simplify this process as much as possible by compiling the tox-easy-bootstrap package for most popular Linux distributions.
To install the tox-easy-bootstrap package, follow the link to the project repositories, select your distribution and follow the simple instructions to add the repository and install the package on your system.
The package installer will automatically receive an up-to-date list of public bootstrap nodes, create a configuration file /etc/tox-bootstrapd.conf and launch the tox-bootstrapd daemonunder a separate system user. Once a week by cron, a special script will update the list of public bootstrap nodes in the configuration file, so you will not need to worry about its relevance in case of a server reboot - “set and forget”.
For cases when a “normally closed” firewall is used on the server, you may need to enter allowing rules for incoming traffic on UDP: 33445 and TCP: 3389,33445 (TCP port: 443 is not used, because the process starts under an unprivileged user) - in order to avoid potential "sabotage", I did not automate this part: Package configuration file /etc/tox-easy-bootstrap.conf
-A INPUT -p tcp --dport 3389 -j ACCEPT
-A INPUT -p tcp --dport 33445 -j ACCEPT
-A INPUT -p udp --dport 33445 -j ACCEPT
allows you to change the default settings (which are suitable for most cases) and, for example, “link” several of your private nodes in case all public nodes are unavailable - as described above, it’s sufficient to connect to any one node in the network to access the second node.
The question of whether to publish data about your node in the list of public nodes remains at your personal discretion - from a technical point of view, a private node is no different from a public one.
Conclusion
The Tox protocol allows you to not only write bots , exchange messages, files or make audio-video calls, but also use it as a basic protocol for other network tasks.
For example, tuntox and toxvpn projects use the Tox protocol to organize P2P host connections for NAT, adding to the list of Full-Mesh VPN solutions .
There are experimental toxmail projects for organizing a mail message, toxscreen for accessing the remote desktop, toxshare for file sharing between a trusted circle of people.
For experiments on writing new applications, you can use wrappers (supported and not very) for other languages and platforms: python , rust , go , node.js - almost unlimited scope for new ideas and projects.
References
- toxstats.com , tox.viktorstanchev.com , stats.letstox.com - Tox network statistics.
- Draft tox protocol specification - toktok.github.io .
- The project for building binary versions of clients and libraries is tox.pkg on github .
- tox.chat - official site of the Tox project.
