December 17, 2013 at 07:04The basics of developing an I2P network client. Part 2
In a previous article , the tasks necessary to build an I2P router, able to take part in the network, including interaction with other routers via the normal Internet, building tunnels of various types and collecting information about other network nodes, were considered. Despite the importance of these tasks, the I2P client, which performs only the functions of a router, is, from the point of view of the user, a “thing in itself” because it does not do anything interesting to the user. This article focuses on application layer protocols designed to transmit user data over an I2P network.
If the I2P routing issues are more or less logically addressed in the official documentation, then application protocols are a mishmash of various ideas, which mean that everyone is free to implement their own protocol for their own application, using destination points as addresses. However, this does not bring us closer to the implementation of our own client, since the existing network resources already use some kind of protocols and any new implementation should be able to work with them. As an example, you can see the description of "garlic", from this page it’s only possible to understand how to pack “garlic” and how to encrypt it. What is transmitted between Alice and Bob, and also how Bob knows that the answer should be sent to Alice, is completely incomprehensible. It also contains a statement that, in order to confirm delivery, in one of the “garlics” a DeliveryStatus message is transmitted indicating the sender in the instructions for delivery, thereby revealing the sender on which the sender is sitting. Of course, this is not so. Unfortunately, the only way to find out how things really are was to analyze the traffic generated by the official Java client.
We will examine this issue in more detail, somewhat modifying the original example and making it more practical. Suppose that Vasya Pupkin accesses the Flibusta website, which is now accessible only through I2P, while Vasya has his own website that offends someone’s feelings, and therefore is located in I2P. To do this, Vasya has launched a router that constantly supports at least one outgoing and one incoming tunnel. Vasya created a separate destination for his website and published its address everywhere. To contact Flibusta, Vasya already knows her address, and his router knows her LeaseSet and can send a message there, the only problem is that Vasya needs to get an answer from Flibusta, and for this she needs to know Vasya's address. To this end, another destination point is created on the Vasin router. serving as the return address for all connections initiated by Vasya. Not only for Flibusta, but also for all other sites. If necessary, you can create several such return addresses, but then you will also have to build LeaseSets with different disjoint sets of tunnels.
To transfer data between destination points, I2NP Garlic - “garlic” message is used. The messages themselves are transmitted and encrypted between the routers, using a separate pair of encryption keys for this, the public key of which is transmitted to the LeaseSet. This key does not coincide with the public key of the router, and unlike the latter, it is generated a new one at each start. Inside the message consists of “garlics,” each of which consists of an I2NP message and instructions for its delivery (4 types):
As a rule, “garlics” consisting of two “garlics” are used. The first is the I2NP message DeliveryStatus with the message number of the “garlic” itself and delivery to one of the incoming tunnels of the sender’s router. Used to confirm the delivery of all "garlic" to its intended purpose. The second is an I2NP Data message containing the transmitted data itself, delivered to the destination. Sometimes there is a third “garlic” - the LeaseSet of the destination (not the marshigator) sender. In our example, this is LeaseSet of Vasya’s return address. Such a “garlic” is present in two cases: in the very first message and when the LeaseSet changes. This is done so that the router will know how to send a response to the sender, otherwise you would have to request the appropriate LeaseSet from the floodfill routers,
The question arises, how does Flibusta know that the answer should be sent to Vasya, and not to Petya or anyone else who is contacting her at that very moment. Even if it were known that the “garlic” came from Vasya, which, of course, is not the case, it would not help much, since Vasya’s router has its website and, possibly, much more . It turns out that this information should be contained within the data transmitted in the Data message, which indicates an unsuccessful design of the entire system, since it does not allow isolation of protocols of different levels from each other. In other words, the destination address must be present both in the data itself and in the protocol for transmitting this data.
Initially, the I2CP protocol was developed exclusively for the exchange between various applications and the router - its messages should not be sent to the I2P network itself. However, the contents of the SendMessageMessage and MessagePayloadMessage messages are transmitted over the network inside the I2NP Data messages, and represent gzip-archived data with a special way the view header is changed.
0x1F 0x8B 0x08 - gzip prefix
1 gzip flags
byte 2 TCP or UDP
bytes sender port 2 TCP or UDP bytes port
1 gzip additional flags
byte 1 protocol type byte: 6 - streaming, 17 - datagram (datagram), 18 - “raw”
Thus, each Data message transmitted through garlic will always start with 0x1F 0x8B 0x08 and first of all it is unpacked by gzip and, depending on the type of protocol, processed by the corresponding implementation.
The stream protocol is similar to the TCP protocol and guarantees the sequence of data transmission. Messages consist of a header and data itself. The type of message is determined by the flag field; similarly to TCP, there are SYN / FIN flags for establishing and disconnecting a connection. Unlike TCP, up to 255 NACKs can also be present - numbers of missed messages with the requirement to resend them, which is more typical for packet protocols and requires a more complex implementation. Also contains the standard TCP fields: 4-byte ports of the sender and receiver, called streams, sequence numbers and exposure numbers.
When a connection is established, messages are exchanged with the SYNCHRONIZE flag set, and the FROM_INCLUDED and SIGNATURE_INCLUDED flags must be set. The first means that the header contains the full 387-byte I2P address, and the second that the entire message is signed by the sender's I2P private key. In this way, the parties recognize each other's I2P addresses, and verification of the signature ensures that these addresses are real. In other words, after connecting to the address of Flibusta, Vasya can be sure that this is Flibusta, and Flibusta will recognize Vasya’s return address.
Thus, the stream protocol interface can be implemented in the form of ordinary sockets, which allows the use of I2P in network applications with minimal changes.
If the I2P routing issues are more or less logically addressed in the official documentation, then application protocols are a mishmash of various ideas, which mean that everyone is free to implement their own protocol for their own application, using destination points as addresses. However, this does not bring us closer to the implementation of our own client, since the existing network resources already use some kind of protocols and any new implementation should be able to work with them. As an example, you can see the description of "garlic", from this page it’s only possible to understand how to pack “garlic” and how to encrypt it. What is transmitted between Alice and Bob, and also how Bob knows that the answer should be sent to Alice, is completely incomprehensible. It also contains a statement that, in order to confirm delivery, in one of the “garlics” a DeliveryStatus message is transmitted indicating the sender in the instructions for delivery, thereby revealing the sender on which the sender is sitting. Of course, this is not so. Unfortunately, the only way to find out how things really are was to analyze the traffic generated by the official Java client.
Garlic data transfer
We will examine this issue in more detail, somewhat modifying the original example and making it more practical. Suppose that Vasya Pupkin accesses the Flibusta website, which is now accessible only through I2P, while Vasya has his own website that offends someone’s feelings, and therefore is located in I2P. To do this, Vasya has launched a router that constantly supports at least one outgoing and one incoming tunnel. Vasya created a separate destination for his website and published its address everywhere. To contact Flibusta, Vasya already knows her address, and his router knows her LeaseSet and can send a message there, the only problem is that Vasya needs to get an answer from Flibusta, and for this she needs to know Vasya's address. To this end, another destination point is created on the Vasin router. serving as the return address for all connections initiated by Vasya. Not only for Flibusta, but also for all other sites. If necessary, you can create several such return addresses, but then you will also have to build LeaseSets with different disjoint sets of tunnels.
To transfer data between destination points, I2NP Garlic - “garlic” message is used. The messages themselves are transmitted and encrypted between the routers, using a separate pair of encryption keys for this, the public key of which is transmitted to the LeaseSet. This key does not coincide with the public key of the router, and unlike the latter, it is generated a new one at each start. Inside the message consists of “garlics,” each of which consists of an I2NP message and instructions for its delivery (4 types):
- Local. The message is for the router itself. Usually someone's LeaseSet.
- Destination point. The message is intended for the destination connected to the router. This is the only way to send data to the destination.
- Tunnel. The message is intended to be sent to the specified incoming tunnel, starting on the specified router. Used to confirm delivery of garlic.
- The marchoutizer. The message is intended to be sent to another router. Extremely dangerous from a security point of view, if the specified router is different from its own or trusted. In practice, not met.
As a rule, “garlics” consisting of two “garlics” are used. The first is the I2NP message DeliveryStatus with the message number of the “garlic” itself and delivery to one of the incoming tunnels of the sender’s router. Used to confirm the delivery of all "garlic" to its intended purpose. The second is an I2NP Data message containing the transmitted data itself, delivered to the destination. Sometimes there is a third “garlic” - the LeaseSet of the destination (not the marshigator) sender. In our example, this is LeaseSet of Vasya’s return address. Such a “garlic” is present in two cases: in the very first message and when the LeaseSet changes. This is done so that the router will know how to send a response to the sender, otherwise you would have to request the appropriate LeaseSet from the floodfill routers,
The question arises, how does Flibusta know that the answer should be sent to Vasya, and not to Petya or anyone else who is contacting her at that very moment. Even if it were known that the “garlic” came from Vasya, which, of course, is not the case, it would not help much, since Vasya’s router has its website and, possibly, much more . It turns out that this information should be contained within the data transmitted in the Data message, which indicates an unsuccessful design of the entire system, since it does not allow isolation of protocols of different levels from each other. In other words, the destination address must be present both in the data itself and in the protocol for transmitting this data.
I2CP Protocol Data
Initially, the I2CP protocol was developed exclusively for the exchange between various applications and the router - its messages should not be sent to the I2P network itself. However, the contents of the SendMessageMessage and MessagePayloadMessage messages are transmitted over the network inside the I2NP Data messages, and represent gzip-archived data with a special way the view header is changed.
0x1F 0x8B 0x08 - gzip prefix
1 gzip flags
byte 2 TCP or UDP
bytes sender port 2 TCP or UDP bytes port
1 gzip additional flags
byte 1 protocol type byte: 6 - streaming, 17 - datagram (datagram), 18 - “raw”
Thus, each Data message transmitted through garlic will always start with 0x1F 0x8B 0x08 and first of all it is unpacked by gzip and, depending on the type of protocol, processed by the corresponding implementation.
Streaming protocol
The stream protocol is similar to the TCP protocol and guarantees the sequence of data transmission. Messages consist of a header and data itself. The type of message is determined by the flag field; similarly to TCP, there are SYN / FIN flags for establishing and disconnecting a connection. Unlike TCP, up to 255 NACKs can also be present - numbers of missed messages with the requirement to resend them, which is more typical for packet protocols and requires a more complex implementation. Also contains the standard TCP fields: 4-byte ports of the sender and receiver, called streams, sequence numbers and exposure numbers.
When a connection is established, messages are exchanged with the SYNCHRONIZE flag set, and the FROM_INCLUDED and SIGNATURE_INCLUDED flags must be set. The first means that the header contains the full 387-byte I2P address, and the second that the entire message is signed by the sender's I2P private key. In this way, the parties recognize each other's I2P addresses, and verification of the signature ensures that these addresses are real. In other words, after connecting to the address of Flibusta, Vasya can be sure that this is Flibusta, and Flibusta will recognize Vasya’s return address.
Thus, the stream protocol interface can be implemented in the form of ordinary sockets, which allows the use of I2P in network applications with minimal changes.