Decentralization: A Big Problem for Blockchain
Hello, Habr! I present to you the translation of the article "Decentralization: The Big Problem For Blockchain".
Decentralization is one of the keywords of blockchain technology: companies and websites have appeared that include this word in their name.
Decentralization was touted as the most advanced fintech feature. The abbreviation DLT (Decentralized Ledger Technology) has become synonymous with blockchain in the authorized Fintech environment.
Few people realize that decentralization is a problem in itself, and blockchain technology has stopped for many years.
Let me explain:
In the 1960s, computer systems were centralized or configured as a star network topology. Only in the early 1970s, the need to connect computers from several manufacturers became urgent.
At that time, the nodes of several existing communication networks were organized hierarchically, but from the very beginning the protocols implemented in the nodes of ARPA, RPCNET, PISA and other innovative networks prior to the Internet were developed with the general idea that the Central node or body should control, manage, be the center, or own the network.
In other words, we knew that a centralized multi-level star network, with its inherent bottlenecks, would not be able to satisfy even the needs of thousands of users in the 1970s. We also suggested that by reducing the number of bottlenecks resulting from decentralization, the problem would be reduced, but not resolved. We knew that a solution should use a distributed peer-to-peer model.
Since many organizations will be involved in providing nodes, channels, and unreliable hardware and software, we assumed that the network was unreliable. We did not know how a consistent dataset or even a single transaction can be supported in several databases via an untrusted network, when any node can generate a message, or in fintech terminology, a financial transaction. The problem was further exacerbated by the presence of malicious players.
In general terms, we recognize that a network is decentralized when network management is shared by a subset of network nodes.
A network is distributed when all nodes share responsibilities equally and run the same node software.
Decentralized (permitted and leader-based) networks were often extensions of centralized networks arising from application requirements, for example, in the fintech industry.
The blockchain network software (basic system software) should not be developed in accordance with the requirements of the application, as they will change. We did not design the network predecessors of the Internet and the Internet itself, based only on the requirements of the 1970s applications. We could not predict which industries will develop based on the ability to exchange information globally.
Likewise, the core blockchain network should be as general, flexible and scalable as possible. Permissions, client-server, and private network requirements can then be considered as special cases of a distributed network, for example, using the concept of virtual private blockchain networks.
Distributed networks are more likely to be independent of any particular physical structure. Nodes can be dynamically connected to each other, and random connection procedures can be used. Consensus solutions implemented in distributed networks can also be invalid, managed by the majority and recursive.
Distribution, not decentralization, should have been the main goal of cryptocurrency design.
The failure to examine distributed consensus agreements is partly due to the 1982 formulation of the problem of Byzantine generals, which models how information integrity can be maintained in an unreliable environment. The problem of Byzantine generals has been studied by researchers for more than thirty years.
The analogy of several allied army divisions holding the besieged city suggested that no one on the field could be trusted to deliver a message, and that some of the generals themselves could not be trusted in issuing a command.
However, the wording of the army analogy implied the presence of at least two classes of troops: generals and soldiers.
Then, some people limited their thinking to a more specific case, when the generals gave commands that needed to be transferred to other generals and protected from interference by intruders or traitors.
In the end, this approach led to a limited definition of the consensus problem.
Leader-based consensus models such as Paxos and Raft were taught at universities and used as models by blockchain network developers.
As a result, practical solutions to the problem of Byzantine generals focused on various methods of selecting a leader node that would send a block of verified information to all other nodes instead of trying to reach consensus on the contents of the block.
Consensus on which node should be the current leader does not solve the trust problem. The current leader must be trusted. He must do the work of checking and assembling the blocks in a fair manner.
Thus, based on the current consensus protocols, the leader must grant some authority: proof of work, proof of stake, proof of ability or proof of something else.
This “evidence” does not guarantee anything more than the personal interest that the leading sites (or sites seeking to be leading sites) can have in the network: the more interest they gain, the less they will be ready to destroy their form of income.
This "evidence" ensures that potential leaders have authority, but do not guarantee that the information collected in the block is correct, or that it has been verified by most nodes.
We saw more than 60 proposed solutions based on leadership models for various blockchain implementations. They suffer from a common error: one node decides what each other node will store in the blockchain. The result is almost the opposite of what is required.
Leader-based protocols have the following disadvantages:
Reflecting on the analogy with the problem of reaching a common solution in an unorganized and unreliable environment, we could use an army analogy without ranks, but it would not be very intuitive.
A better analogy would be the task of determining the daily closing price of the stock exchange. By this analogy, many buyers and sellers determine the daily closing price of stocks using the stochastic process, without any specific decision-maker for anyone else.
There is no “right” answer to the stock price on the stock market, but only an agreed daily closing price.
Similarly, in a block, several variables, such as the order of transactions, can determine the final composition of the block. There is no “correct” composition of blocks, only one with which nodes are consistent.
Consensus protocols based on a more distributed analogy could avoid the trend towards centralization and the need for intermediate nodes typical of leader-based protocols.
Examples of network intermediaries:
First of all, it is a matter of cost: if intermediaries do useful work, for example, verify transactions, they need to be rewarded.
This is also a matter of trust: Clients using a network with intermediaries must trust:
Finally, it is a matter of data availability. If the network has a privileged or limited class of nodes that control the blockchain, then most nodes do not have instant access to the current blockchain replica. This can interfere with the development of real-time applications, such as automated trading applications.
Most experts will tell you that the main function of the negotiated protocols is to maintain network security. This view mixes two issues. Security is necessary, but it is a completely different requirement that can be solved in other ways.
However, many developers hold the idea that consensus means choosing a leader, and that consensus is necessary to maintain security.
Looking back, if we were thinking of a better distributed analogy, the study could go in a different direction, offering stochastic approaches, and could lead us to earlier development of better distributed solutions without intermediaries.
Now this is a problem of education, not a technical problem. Most researchers, consultants, and crypto network experts have all the details of PoW, PoS, DPoS, and dozens of alternatives based on the same leader-based model.
Those few solutions that are not based on leaders are not blockchain solutions: these are solutions where each transaction is processed separately.
On the other hand, most people and 95% of companies, according to a recent survey, understand the potential of blockchain technology.
The transition from a decentralized to a distributed model is urgently needed to unleash the true potential of blockchain, solve scalability problems, and run blockchain on any user device without intermediaries.
Decentralization is one of the keywords of blockchain technology: companies and websites have appeared that include this word in their name.
Decentralization was touted as the most advanced fintech feature. The abbreviation DLT (Decentralized Ledger Technology) has become synonymous with blockchain in the authorized Fintech environment.
Few people realize that decentralization is a problem in itself, and blockchain technology has stopped for many years.
Let me explain:
In the 1960s, computer systems were centralized or configured as a star network topology. Only in the early 1970s, the need to connect computers from several manufacturers became urgent.
At that time, the nodes of several existing communication networks were organized hierarchically, but from the very beginning the protocols implemented in the nodes of ARPA, RPCNET, PISA and other innovative networks prior to the Internet were developed with the general idea that the Central node or body should control, manage, be the center, or own the network.
In other words, we knew that a centralized multi-level star network, with its inherent bottlenecks, would not be able to satisfy even the needs of thousands of users in the 1970s. We also suggested that by reducing the number of bottlenecks resulting from decentralization, the problem would be reduced, but not resolved. We knew that a solution should use a distributed peer-to-peer model.
Since many organizations will be involved in providing nodes, channels, and unreliable hardware and software, we assumed that the network was unreliable. We did not know how a consistent dataset or even a single transaction can be supported in several databases via an untrusted network, when any node can generate a message, or in fintech terminology, a financial transaction. The problem was further exacerbated by the presence of malicious players.
Why blockchain technology discourages decentralization
In general terms, we recognize that a network is decentralized when network management is shared by a subset of network nodes.
A network is distributed when all nodes share responsibilities equally and run the same node software.
Decentralized (permitted and leader-based) networks were often extensions of centralized networks arising from application requirements, for example, in the fintech industry.
The blockchain network software (basic system software) should not be developed in accordance with the requirements of the application, as they will change. We did not design the network predecessors of the Internet and the Internet itself, based only on the requirements of the 1970s applications. We could not predict which industries will develop based on the ability to exchange information globally.
Likewise, the core blockchain network should be as general, flexible and scalable as possible. Permissions, client-server, and private network requirements can then be considered as special cases of a distributed network, for example, using the concept of virtual private blockchain networks.
Distributed networks are more likely to be independent of any particular physical structure. Nodes can be dynamically connected to each other, and random connection procedures can be used. Consensus solutions implemented in distributed networks can also be invalid, managed by the majority and recursive.
Distribution, not decentralization, should have been the main goal of cryptocurrency design.
Why we failed
The failure to examine distributed consensus agreements is partly due to the 1982 formulation of the problem of Byzantine generals, which models how information integrity can be maintained in an unreliable environment. The problem of Byzantine generals has been studied by researchers for more than thirty years.
The analogy of several allied army divisions holding the besieged city suggested that no one on the field could be trusted to deliver a message, and that some of the generals themselves could not be trusted in issuing a command.
However, the wording of the army analogy implied the presence of at least two classes of troops: generals and soldiers.
Then, some people limited their thinking to a more specific case, when the generals gave commands that needed to be transferred to other generals and protected from interference by intruders or traitors.
In the end, this approach led to a limited definition of the consensus problem.
Leader-based consensus models such as Paxos and Raft were taught at universities and used as models by blockchain network developers.
As a result, practical solutions to the problem of Byzantine generals focused on various methods of selecting a leader node that would send a block of verified information to all other nodes instead of trying to reach consensus on the contents of the block.
Missing target
Consensus on which node should be the current leader does not solve the trust problem. The current leader must be trusted. He must do the work of checking and assembling the blocks in a fair manner.
Thus, based on the current consensus protocols, the leader must grant some authority: proof of work, proof of stake, proof of ability or proof of something else.
This “evidence” does not guarantee anything more than the personal interest that the leading sites (or sites seeking to be leading sites) can have in the network: the more interest they gain, the less they will be ready to destroy their form of income.
This "evidence" ensures that potential leaders have authority, but do not guarantee that the information collected in the block is correct, or that it has been verified by most nodes.
We saw more than 60 proposed solutions based on leadership models for various blockchain implementations. They suffer from a common error: one node decides what each other node will store in the blockchain. The result is almost the opposite of what is required.
Summary of Shortcomings Based on Leader Protocols
Leader-based protocols have the following disadvantages:
- They do not solve the problem of trust. The leader node can enter erroneous data, intentionally or not, into the information block.
- The awards associated with the work of checking and assembling the blocks create an incentive for the nodes to compete for awards and advance to leadership positions. This incentive tends to create a special class of nodes. The network is becoming a decentralized network. For example, Bitcoin began as a network of partners, where each node could verify transactions and fight for a reward. Today it is a network of two classes (miners and users), which is managed by the owners of large pools.
- When a block assembly is transferred to one node, one of the basic requirements of consensus theory becomes invalid: the agreement is not based on the majority consensus about what information will be stored in the blockchain. The only agreement reached is the method of selecting the leader node.
- A bottleneck is introduced, or a single point of failure: one node must broadcast a block to each other node.
- Efficiency is not the best: large blocks of data are more prone to transmission errors and retransmission of packets of maximum size.
- The redundancy is almost 100%: each transaction included in the block was already received by each node separately when the transaction was originally completed.
Best analogy
Reflecting on the analogy with the problem of reaching a common solution in an unorganized and unreliable environment, we could use an army analogy without ranks, but it would not be very intuitive.
A better analogy would be the task of determining the daily closing price of the stock exchange. By this analogy, many buyers and sellers determine the daily closing price of stocks using the stochastic process, without any specific decision-maker for anyone else.
There is no “right” answer to the stock price on the stock market, but only an agreed daily closing price.
Similarly, in a block, several variables, such as the order of transactions, can determine the final composition of the block. There is no “correct” composition of blocks, only one with which nodes are consistent.
Consensus protocols based on a more distributed analogy could avoid the trend towards centralization and the need for intermediate nodes typical of leader-based protocols.
Examples of network intermediaries:
- Miners, manufacturers or supervisors who voluntarily participate or are involved in the provision of network services,
- Special nodes of federated systems that are interested in the success of the network,
- Nodes selected with some criteria for network management,
- Nodes owned by trusted companies or institutions
- Special players, such as centralized currency exchanges that own user wallets.
What is wrong with intermediaries?
First of all, it is a matter of cost: if intermediaries do useful work, for example, verify transactions, they need to be rewarded.
This is also a matter of trust: Clients using a network with intermediaries must trust:
- That the intermediary does not give preference to specific users or transactions,
- That the mediator was not or will not be captured by the attacker,
- That the intermediary system does not shut down, does not undergo a DoS attack, system failure, or any other reason that will affect or delay client transactions,
- System software and reseller data are reliable, so data integrity and security are guaranteed.
- That they are really connected with a trusted system, and not with a copycat (for example, with some other system that claims to be a trusted intermediary), and
- That no other unpredictable event will happen. Recently, for example, the owner of the Canadian currency exchange Quadriga died or disappeared. As a result, millions of dollars of customer funds are missing.
Finally, it is a matter of data availability. If the network has a privileged or limited class of nodes that control the blockchain, then most nodes do not have instant access to the current blockchain replica. This can interfere with the development of real-time applications, such as automated trading applications.
Is it too late to change the consensus model in the blockchain?
Most experts will tell you that the main function of the negotiated protocols is to maintain network security. This view mixes two issues. Security is necessary, but it is a completely different requirement that can be solved in other ways.
However, many developers hold the idea that consensus means choosing a leader, and that consensus is necessary to maintain security.
Looking back, if we were thinking of a better distributed analogy, the study could go in a different direction, offering stochastic approaches, and could lead us to earlier development of better distributed solutions without intermediaries.
Now this is a problem of education, not a technical problem. Most researchers, consultants, and crypto network experts have all the details of PoW, PoS, DPoS, and dozens of alternatives based on the same leader-based model.
Those few solutions that are not based on leaders are not blockchain solutions: these are solutions where each transaction is processed separately.
On the other hand, most people and 95% of companies, according to a recent survey, understand the potential of blockchain technology.
The transition from a decentralized to a distributed model is urgently needed to unleash the true potential of blockchain, solve scalability problems, and run blockchain on any user device without intermediaries.