Wallet synchronization with the Bitcoin network

    Let's talk about synchronizing digital wallets with the Bitcoin network. Denote the main approaches, the principles of their work and complexity. The material can be useful not only to developers, but also to ordinary users to familiarize themselves with the features of the device of digital wallets and aspects of safe work with wallets.

    According to the material of this article, one of the lectures of the online blockchain course “ Synchronizing wallets with the Bitcoin network ” is recorded .

    Basic wallet functions

    Let's start with the basic functionality of a digital wallet:

    • backup private keys;
    • payment acceptance;
    • balance display;
    • display transaction history;
    • sending payments.

    To back up your private keys, you only need the module responsible for storing and managing keys. However, in the case of the other four functions, you will definitely need to work with the Bitcoin network. After all, in order to read and write a common transaction database, you must at least become one of the nodes of this network and follow the processing and data exchange specifications in it. Thus, it is necessary to have data on all transactions in which the addresses of a certain wallet participated. Data on the current state of the network will also be needed to draw up a new transaction and make a payment.

    The most important thing that a digital wallet user must reliably know when receiving and sending payments is the status of their transactions. Obviously, the transaction is considered incomplete if the transaction was sent, but did not receive a full confirmation. As a result, the recipient cannot accept payment in order, for example, to deliver a product or provide a service.

    The complexity of working in a distributed network

    Working with a distributed network is quite a complex topic, which usually encounters the following problems. One of them is the way to verify the received data from a distributed network, and the second is to optimize the data storage of this data. These requirements conflict slightly with each other, since a larger amount of stored data allows for a more complete check of new transactions and blocks. There is no perfect solution to these problems, it is impossible to process transactions both reliably and with minimal resources, so you have to find some balance, a compromise between simplicity and security.

    Another problem is this. To get transaction history at a specific address on a full host, you will have to go through the whole blockchain (from genesis to last known), which is quite a long time, since this is a large amount of data, and a rough search is a little effective approach. Therefore, you can use certain approaches to optimize queries. For this purpose, special add-ons are used above the usual network node, which help index the blockchain and cache some data to speed up the search for popular metrics. Any blockchain explorer, for example blockchain.info, works on this principle, which fairly quickly returns data on almost any address, on any transaction, etc.

    So, there are several fundamentally different approaches to the interaction and synchronization of a digital wallet with a distributed network. Let's consider them.

    Approaches to synchronize the wallet with the payment network

    There are three main approaches. The first assumes that the wallet itself is a complete Bitcoin network node. In the second approach, a so-called trusted node is used. In fact, this is a foreign network node, but the user of the wallet trusts him. And the third approach involves direct interaction with the rest of the network nodes, but in a lightweight version, using the Simplified Payment Verification (SPV) method. This technique allows a fairly high degree of reliability to verify the confirmation of transactions, but does not resort to running a full network node. Let's take a closer look at each of these synchronization approaches.


    Work with a complete network node

    In the first case, everything is quite simple. Your wallet has a module that stores the entire blockchain, and a module that communicates with other nodes of the Bitcoin network. Therefore, you can always check the new incoming transaction: whether it really complies with the rules of the protocol, whether it does not do double spending, etc. You have complete control over the verification of data.

    The disadvantage of this approach is the need to allocate a large amount of disk space for storing a full blockchain, contain a network node in the mode of smooth operation and timely software update.

    This method is used by ordinary users who regularly accept and send payments for large sums, which requires a high level of reliability. In addition, all major services that provide the ability to centrally store bitcoins and online access to the wallet, necessarily use this approach. Because there are high security requirements, as well as independent work with Bitcoin.

    It is important to understand that the user of the centralized bitcoin repository is completely dependent on this service and the correct operation of the network node that is used by this service. In fact, if coins are stored on such a service, they do not belong to the user but to the service.

    Note that using this approach is one of the most simple and reliable, even for the average user. To do this, you just need to take the source code or ready binaries, for example, Bitcoin Core and run the software. A significant drawback can be considered only computer requirements.

    Work with a trusted host

    The difference of this approach from the previous one is in the separation of the wallet logic and the transaction verification logic. There is a so-called trusted Bitcoin host. Roughly speaking, this is the most common complete Bitcoin network node, which the user trusts him to verify their transactions. For example, it can be a host that is supported by his friend, and the user can trust him with the verification of his transactions.

    Very often, this approach is used in mobile digital wallets. Suppose a company that develops a mobile wallet supports trusted nodes and promises that it will ensure correct transaction verification. In this case, users trust the company to the developer of a digital wallet, in fact, they promise promises. But it should be understood that the user can independently store their private keys and authenticate transactions on their device. However, the user does not check the transaction confirmation status, but asks the trusted site.

    What are the advantages and disadvantages of this approach? As in the previous case, albeit indirectly, you can influence the verification of transactions in the Bitcoin network, because the user has a trusted network node. The advantage is that it frees itself from having to keep a full copy of the blockchain on its device. Nevertheless, dependence on a trusted host is a disadvantage, because in the event of its refusal, the user cannot always switch to another trusted host. Since transaction verification takes place on a remote server, you need to make sure that the data transfer channel between the wallet and the server on which the verification takes place is reliable. With this approach, there are threats of such attacks as “Man in the middle”.

    When a trusted host is used, private keys are stored separately from wallet addresses. In this case, the node itself cannot reliably know the changes to which addresses it needs to monitor. This means that the digital wallet should from time to time communicate to the trusted node a list of addresses that interest it.

    The disadvantage of this approach is the dependence of the digital wallet on the trusted site, in particular in terms of access to relevant data of the payment system. As soon as the user loses access to the trusted node or the node refuses to be serviced, the user loses connection to the network. Accordingly, the user loses the ability to receive and send payments. To reduce the risk of losing access to a trusted host, you can select multiple trusted hosts. This means that in normal conditions the user connects and works with one trusted node, but if a loss of access occurs, the digital wallet automatically connects to another trusted node. In fact, at each connection, the wallet randomly selects one node from the trusted list.

    This method is mainly used by mobile wallets, because it allows both to verify transactions and simplify the work of a mobile application. Examples of such mobile wallets for Bitcoin are Mycelium and Coinomi. Distributed Lab has also developed and maintains a wallet called Bitxfy, where this approach is applied.

    Work with SPV network node

    The third common approach to synchronizing a digital wallet with a Bitcoin network is a lightweight network node. This is the Simplified Payment Verification (SPV). In this approach, you may not need to run your full sites or select trusted ones. It is assumed that the digital wallet communicates directly with other nodes on the network: in fact it selects several dozen foreign full nodes and maintains connection with them. The difference is that a lightweight node communicates with other nodes on an equal footing, exchanging p2p messages with them, but does not store the entire transaction history. Here SPV is used to check incoming transactions. In fact, the lightweight node conducts only a part of the most important checks of those that conducts a complete network node.

    How does the work in this case? Instead of receiving blocks, as the full node does, the SPV node receives only block headers that are much smaller (80 bytes). The header contains the necessary data to verify the fact of confirmation of a specific transaction in this block. More specifically, the block header contains the Merkle Root field, which contains the hash value from all transactions in the block, calculated according to the Merkle Tree scheme.


    The Merkle Tree scheme allows you to check whether the hash value of a particular transaction is included in Merkle Root, without having all the other block transactions. Thus, having a transaction, a block header and a Merkle Branch for this transaction (can be requested from the full node), the digital wallet can verify that the transaction was indeed confirmed in a particular block.

    Why it can not be considered a full transaction verification? Of course, the user becomes independent of specific network nodes and can communicate directly with the distributed network. To do this, he does not need to store the entire blockchain. But the disadvantage is that the lightweight client does not independently check the transaction completely, because you do not have all the necessary data for this (for example, unused coins). He checks only the fact of confirmation of the transaction by the validators - that is, he is convinced that this transaction has been completely checked by the other nodes and the owners of the majority of the computing power of the network (miners). In this case, reliability again comes down to the trust that most computing power is used for fair work. In other words, you do not check the transaction yourself, but trust it to the majority.

    The peculiarity of using digital wallets that implement the approach of an SPV node is the need to maintain a network connection with a large number of independent full network nodes. This is necessary to minimize the likelihood that the node will connect and receive data only from the attacker's nodes. Then the attacker can impose an alternative (fake) state of the transaction history on the user. As a result, the victim may accept a payment from the transaction, which is confirmed by the attacker, although in fact it does not. In order to increase the chances of obtaining actual data on the real state of the network, the user should maximize the number of independent nodes for communication. The more of them, the less likely they are in collusion against the user.

    As you already understood, SPV nodes are demanding for stable operation of a network connection, however, they are often used even in mobile applications. This approach to synchronization with the payment network is quite widespread and is used in a number of popular Bitcoin wallets. Among them are Bitcoin Wallet, Electrum, and Bread Wallet.

    Frequently asked Questions

    - Where can I find complete sites that can be trusted?

    It's hard enough to answer which nodes you can trust. This is a matter of personal preference. There are services that provide trusted information, such as Blockchain.info, - rarely anyone trusts them, especially when it comes to large amounts. You need to be very careful in choosing a trusted host, because it is an intermediary between the user and the payment network. Once Blockchain.info failed for two days due to problems with the DNS provider, Wallets that used Blockchain.info as a trusted host could not synchronize with the Bitcoin network and at least work somehow.

    - What guarantees does the wallet team give and what is its responsibility?

    This question is more legal than technical. Any software that is distributed officially has a “Terms of Use”, where it is stated that you can submit to the developers if certain problems arise in their software. “Terms of Use” differ in different applications. As always, the user answers the question whether he trusts developers or not before starting to use their product.

    - Who decides if the node is trusted?

    Only the user who owns the wallet. The value of trusted sites is a significant simplification of the wallet. You select a specific node that you trust transaction verification. It takes all the burden of performing verification operations on itself, and you trust the data on the state of the network, which it sends to you. This is a matter of personal trust and nothing more.

    - How many trusted sites can there be?

    You can select any number of trusted sites to minimize the risk of losing access to the network. For example, three nodes. If you have problems with access to the first, then you connect to the second and continue to work with him on the same principle.

    - What happens to the coins if the servers on which they are stored will be completely or partially destroyed by an earthquake?

    Here it is necessary to say not about the coins, but about the storage of the keys that are used to access these coins. Consider two options for one situation. If destroyed the server that provided access to the Bitcoin network. It will not affect you in any way. You temporarily lose access to the distributed database, but you still have the keys that allow you to spend these coins. If the server where your keys were stored was destroyed, and you do not have a backup copy of these keys, then you will not be able to access your coins.

    - Is it safe to put a wallet, put coins on it, write a seed phrase on paper and delete it?

    Essentially, yes. You need to save the seed phrase - this is a sufficient minimum. The blockchain stores information about which coins are stored at which addresses. You need to keep your private keys to access these coins. In your case, this is a seed phrase.

    - Is it possible to put some add-on on the blockchain locally in order to be able to analyze the data in the blockchain?

    You can add any software that will analyze the blockchain. This may be software written by the user, or there may be a ready-made solution. It all depends on your goals. If you want to optimize queries for finding specific transactions associated with specific addresses, as the blockchain explorer does, you can use software that will index the blockchain and cache certain data. An example of such software is BitCore (supported by BitPay). BitCore is open source and you can simply use it with your complete node.

    Also popular now: