About µTP in new versions of µTorrent: what is it, how, why?
Traditionally, most P2P applications have used TCP to exchange data. About the fact that µTorrent begins to use the new protocol based on UDP, it was already mentioned on the hub ( one , two ). In this post, the new µTP protocol is described in more detail, including its tuning and the ability to disable it. Details are described in such a way as to make it clear to people far from network protocols.
Update : The official documentation for the protocol: www.bittorrent.org/beps/bep_0029.html
A few words about TCP and UDP. The first stands for Transmission Control Protocol, or flow control protocol. It is convenient in that it gives the program using it a guarantee that the data reaches the addressee intact, in full and in the order in which they were sent. Its use requires a preliminary installation of the connection and its closure at the end. This is the creation of a “pipe” through which data will be exchanged. UDP does not provide any guarantee that the data will arrive, or that it will arrive in the correct order. It only allows you to send a small block of data (datagram) from one address to another. All the work of checking the delivery, and if necessary, re-sending, rests with the program itself.
Since the torrent client is already doing this, this is not a big problem. The fact is that the data sent through the TCP “pipe” in the process is broken into pieces (“packets”), each of which is sent independently. At the same time, one packet can go one route, another - another, the last piece can come first, the first - generally get lost along the way. Therefore, each participant in the "pipe" (from the operating system to the routers) is forced to keep a buffer in which it collects individual packets, checking integrity and order, and requiring forwarding if some of the packets did not reach.
At the same time, if the sender sits on a wide channel, and the receiver on the modem, the first immediately sends a large block of data, which can quickly reach the provider of the second, and slowly seep into the modem. At this time, the first, having not received confirmation of receipt, will forward part of the pieces again. Once again, and again, as a result, the provider's backbone is clogged with this unnecessary forwarding. One of the main goals of uTP is to eliminate this extra burden on providers from P2P traffic.
Latently uTP appeared in µTorrent version 1.8, but was able to accept only incoming uTP connections, it did not know how to initiate them itself. This was first learned by the alpha version 1.9, then it became possible to enable this in the new versions 1.8 using the bt.transp_disposition key. Its value changed from version to version, but now it has settled on the following bit flags:
1 - allow initiating outgoing TCP connections,
2 - allowing initiating outgoing uTP connections,
4 - allowing accepting incoming TCP connections,
8 - allowing accepting incoming uTP- connections
Thus, 13 (1 + 4 + 8), the default value in recent versions 1.8, means the ability to accept all kinds of connections, but to install only TCP yourself. 15 (the default value is 2.0) allows all types of both outgoing and incoming connections. To disable uTP at all (if it causes any problems), you need to put 5(1 + 4). Is it worth it to put 15 in 1.8 - a moot point, they write on the official forum that uTP support in version 2.0 is much better, so the speed in 1.8 can be worse than over TCP.
Version 2.0 also introduced support for UDP trackers. For the tracker in general, TCP is very redundant and requires a lot of extra resources with connection setup / close packages, so UDP is good for open trackers. For example, recently the anirena tracker is very buggy over TCP and answers far from the first time, DHT is often prohibited there, and the UDP tracker works fine. But for closed trackers, it is not suitable, because does not allow to send passers to identify in this way swinging.
Back in 2.0 there will be a method to bypass some NAT ( STUN), which will help connect more NAT sufferers, although it will not work in all cases.
Of all 2.0 innovations, I personally was most pleased with TCP Rate Control. It allows you to adjust the speed and TCP connections so that they do not interfere with other applications and minimize unnecessary packet forwarding, which is described above. Previously, this was achieved by installing the cFos driver, in which it was possible to put a low priority on the torrent, high on the browser, and with version 2.0 this is no longer required. This is controlled by the bt.tcp_rate_control option , if it’s more important for you that the torrent does not interfere with other Internet applications - it should be turned on, if the maximum speed of the torrent is important - it makes sense to turn it off, they write on the official forum that this sometimes increases the speed.
I note that in uTP this is always done, even in version 1.8, and the speed of TCP traffic is adjusted to the channel load only in 2.0. In uTP, the response time from peers with which data is exchanged is constantly measured. as soon as this "ping" begins to increase, long before the start of packet loss, µTorrent slows down.
Due to this, while the channel is free - it is used to the full. as soon as, for example, another application (browser) starts loading the channel, the response time starts to increase in µTorrent, and it automatically releases the channel for the browser. As soon as the ping has returned (the channel is free again) - µTorrent increases the load of the channel. At the same time, there is much less unnecessary packet forwarding than if it happened at the TCP level
However, there is an interesting point here that with it or via uTP, the outgoing channel can be blocked by 95%, and without it only with TCP - by 100%, but this 5% difference may turn out to be the very unnecessary forwarding of the same packets, that the channel clogs, but does not bring real benefits, so imho is not always when the channel is slightly overshot, this means that the new version is worse.
Even in discussions, the key often flashes net.calc_overhead. If it is turned on, then when adjusting the speed, service traffic is also taken into account - requests for blocks, confirmations, notifications of which blocks anyone has, etc. If disabled, only the data blocks themselves are considered. Therefore, it was previously advised to limit the outgoing channel in the torrent to 80-90% of the real one, otherwise it was completely clogged by the blocks being sent, and notifications of receipt of blocks and requests for new ones did not have time to pass, so the download speed was seriously affected. Again, on wide channels, some write that when you turn off this option, the speed is slightly higher, but maybe it's a beta bug and everything will be fine in the release.
There is still such a moment in which I am not 100% sure, but it seems that there is more service traffic in uTP. For in each small datagram it is necessary to transmit what block it is, from which torrent, and TCP can send large blocks without signing each piece. However, there will be official traffic of TCP itself, so it’s not a fact that it is much more “economical”.
One cannot fail to mention such a phenomenon as “shaping” or “cutting” of P2P traffic by some providers. For Russia, this (so far?) Is not relevant, but on open trackers, where there are a lot of peers from around the world, the uTP speed is much higher.
On the other hand, not all network hardware — modems, routers — and not all software relied on so much UDP traffic, so some users experience glitches with it. For example, with speed - when it periodically suddenly drops to zero, then recovers again, or floats from zero to maximum. Apparently somewhere in the network environment, a certain buffer associated with UDP is overflowing, damn. Again, on the official forum you can search for compatibility with different software and hardware in case of problems.
Other "advanced" options that you can pay attention to:
bt.connect_speed - how many new connections can be established per second (worth increasing, I have 80),
net.max_halfopen- Much has been written about this, and it is worth changing with the tcpip.sys patch, although this is no longer important with the uTP protocol.
net.utp_target_delay is a kind of target “ping” when adjusting connections, in some cases, when it increases somewhere to 400-500, the speed becomes better.
peer.disconnect_inactive_interval - after how many seconds the connection with the peer, with which there is no data exchange, is closed, it is more important for open trackers where there are more people and “bad” peers, or in case of network glitches - in order to quickly determine the disconnection and reinstall it. in some cases it makes sense to lower to 90-120.
Version 2.0, although beta is quite stable, can be downloaded here: forum.utorrent.com/viewtopic.php?id=60602
According to personal feelings, in the old alphas 1.9 when downloading from open trackers, if before that the incoming channel was loaded somewhere at 1/3, with uTP it started loading somewhere at 2/3. When downloading from closed trackers, the channel used to load so that the browser starts to slow down, and in 2.0 everything flies.
Update : The official documentation for the protocol: www.bittorrent.org/beps/bep_0029.html
A few words about TCP and UDP. The first stands for Transmission Control Protocol, or flow control protocol. It is convenient in that it gives the program using it a guarantee that the data reaches the addressee intact, in full and in the order in which they were sent. Its use requires a preliminary installation of the connection and its closure at the end. This is the creation of a “pipe” through which data will be exchanged. UDP does not provide any guarantee that the data will arrive, or that it will arrive in the correct order. It only allows you to send a small block of data (datagram) from one address to another. All the work of checking the delivery, and if necessary, re-sending, rests with the program itself.
Since the torrent client is already doing this, this is not a big problem. The fact is that the data sent through the TCP “pipe” in the process is broken into pieces (“packets”), each of which is sent independently. At the same time, one packet can go one route, another - another, the last piece can come first, the first - generally get lost along the way. Therefore, each participant in the "pipe" (from the operating system to the routers) is forced to keep a buffer in which it collects individual packets, checking integrity and order, and requiring forwarding if some of the packets did not reach.
At the same time, if the sender sits on a wide channel, and the receiver on the modem, the first immediately sends a large block of data, which can quickly reach the provider of the second, and slowly seep into the modem. At this time, the first, having not received confirmation of receipt, will forward part of the pieces again. Once again, and again, as a result, the provider's backbone is clogged with this unnecessary forwarding. One of the main goals of uTP is to eliminate this extra burden on providers from P2P traffic.
Latently uTP appeared in µTorrent version 1.8, but was able to accept only incoming uTP connections, it did not know how to initiate them itself. This was first learned by the alpha version 1.9, then it became possible to enable this in the new versions 1.8 using the bt.transp_disposition key. Its value changed from version to version, but now it has settled on the following bit flags:
1 - allow initiating outgoing TCP connections,
2 - allowing initiating outgoing uTP connections,
4 - allowing accepting incoming TCP connections,
8 - allowing accepting incoming uTP- connections
Thus, 13 (1 + 4 + 8), the default value in recent versions 1.8, means the ability to accept all kinds of connections, but to install only TCP yourself. 15 (the default value is 2.0) allows all types of both outgoing and incoming connections. To disable uTP at all (if it causes any problems), you need to put 5(1 + 4). Is it worth it to put 15 in 1.8 - a moot point, they write on the official forum that uTP support in version 2.0 is much better, so the speed in 1.8 can be worse than over TCP.
Version 2.0 also introduced support for UDP trackers. For the tracker in general, TCP is very redundant and requires a lot of extra resources with connection setup / close packages, so UDP is good for open trackers. For example, recently the anirena tracker is very buggy over TCP and answers far from the first time, DHT is often prohibited there, and the UDP tracker works fine. But for closed trackers, it is not suitable, because does not allow to send passers to identify in this way swinging.
Back in 2.0 there will be a method to bypass some NAT ( STUN), which will help connect more NAT sufferers, although it will not work in all cases.
Of all 2.0 innovations, I personally was most pleased with TCP Rate Control. It allows you to adjust the speed and TCP connections so that they do not interfere with other applications and minimize unnecessary packet forwarding, which is described above. Previously, this was achieved by installing the cFos driver, in which it was possible to put a low priority on the torrent, high on the browser, and with version 2.0 this is no longer required. This is controlled by the bt.tcp_rate_control option , if it’s more important for you that the torrent does not interfere with other Internet applications - it should be turned on, if the maximum speed of the torrent is important - it makes sense to turn it off, they write on the official forum that this sometimes increases the speed.
I note that in uTP this is always done, even in version 1.8, and the speed of TCP traffic is adjusted to the channel load only in 2.0. In uTP, the response time from peers with which data is exchanged is constantly measured. as soon as this "ping" begins to increase, long before the start of packet loss, µTorrent slows down.
Due to this, while the channel is free - it is used to the full. as soon as, for example, another application (browser) starts loading the channel, the response time starts to increase in µTorrent, and it automatically releases the channel for the browser. As soon as the ping has returned (the channel is free again) - µTorrent increases the load of the channel. At the same time, there is much less unnecessary packet forwarding than if it happened at the TCP level
However, there is an interesting point here that with it or via uTP, the outgoing channel can be blocked by 95%, and without it only with TCP - by 100%, but this 5% difference may turn out to be the very unnecessary forwarding of the same packets, that the channel clogs, but does not bring real benefits, so imho is not always when the channel is slightly overshot, this means that the new version is worse.
Even in discussions, the key often flashes net.calc_overhead. If it is turned on, then when adjusting the speed, service traffic is also taken into account - requests for blocks, confirmations, notifications of which blocks anyone has, etc. If disabled, only the data blocks themselves are considered. Therefore, it was previously advised to limit the outgoing channel in the torrent to 80-90% of the real one, otherwise it was completely clogged by the blocks being sent, and notifications of receipt of blocks and requests for new ones did not have time to pass, so the download speed was seriously affected. Again, on wide channels, some write that when you turn off this option, the speed is slightly higher, but maybe it's a beta bug and everything will be fine in the release.
There is still such a moment in which I am not 100% sure, but it seems that there is more service traffic in uTP. For in each small datagram it is necessary to transmit what block it is, from which torrent, and TCP can send large blocks without signing each piece. However, there will be official traffic of TCP itself, so it’s not a fact that it is much more “economical”.
One cannot fail to mention such a phenomenon as “shaping” or “cutting” of P2P traffic by some providers. For Russia, this (so far?) Is not relevant, but on open trackers, where there are a lot of peers from around the world, the uTP speed is much higher.
On the other hand, not all network hardware — modems, routers — and not all software relied on so much UDP traffic, so some users experience glitches with it. For example, with speed - when it periodically suddenly drops to zero, then recovers again, or floats from zero to maximum. Apparently somewhere in the network environment, a certain buffer associated with UDP is overflowing, damn. Again, on the official forum you can search for compatibility with different software and hardware in case of problems.
Other "advanced" options that you can pay attention to:
bt.connect_speed - how many new connections can be established per second (worth increasing, I have 80),
net.max_halfopen- Much has been written about this, and it is worth changing with the tcpip.sys patch, although this is no longer important with the uTP protocol.
net.utp_target_delay is a kind of target “ping” when adjusting connections, in some cases, when it increases somewhere to 400-500, the speed becomes better.
peer.disconnect_inactive_interval - after how many seconds the connection with the peer, with which there is no data exchange, is closed, it is more important for open trackers where there are more people and “bad” peers, or in case of network glitches - in order to quickly determine the disconnection and reinstall it. in some cases it makes sense to lower to 90-120.
Version 2.0, although beta is quite stable, can be downloaded here: forum.utorrent.com/viewtopic.php?id=60602
According to personal feelings, in the old alphas 1.9 when downloading from open trackers, if before that the incoming channel was loaded somewhere at 1/3, with uTP it started loading somewhere at 2/3. When downloading from closed trackers, the channel used to load so that the browser starts to slow down, and in 2.0 everything flies.