Bitcoin in a nutshell - mining

Published on January 23, 2017

Bitcoin in a nutshell - mining

    Even people who are infinitely far from the topic of cryptocurrencies have most likely heard about mining. Probably you, dear reader, thought about how to turn on your gaming Pentium 4 at night, and wake up already rich in the morning.


    But, as often happens in the world of blockchain, there are a lot of those who heard, but there are only a few who really understand the process from beginning to end. Therefore, in the last chapter I tried to cover as much detail as possible all the subtleties, from the technical implementation of PoW, ending with the profitability of mining on video cards.


    mining_meme


    Book



    Table of content


    1. Explain me like I'm five
    2. Sky is the limit?
    3. Reward
    4. Hard challenge
    5. Technical side
    6. 2 Blocks 1 Chain
    7. Hardware
    8. Conclusion
    9. Links

    Explain me like I'm five


    Mining , also  mining  (from the  English  mining  - mining) - activities to maintain a distributed platform and create new blocks with the ability to receive rewards in the form of issued currency and commission fees in various  cryptocurrencies , in particular  Bitcoin . The calculations are required to provide protection against reuse of the same currency units, and the connection of mining with emissions stimulates people to spend their computing power and support the operation of networks - Wikipedia

    If on the fingers, then mining is a critical process for Bitcoin, which consists in creating new blocks and pursuing two goals at once. The first is money supply production. Each time a miner creates a new block, he is rewarded for this with the Nth number of coins that he then spends somewhere, thereby launching new funds into the network.


    The second, and much more important goal, is to ensure the operation of the entire network. Surely, while reading the previous articles, you have already asked yourself the questions "Who is the person who checks transaction scripts?" or "If I specify an already used output as an input, at what point will they notice it?" .


    So, all these actions are performed primarily by miners. Well, in fact, each member of the network to one degree or another ensures its security. Synchronizing Bitcoin for so long, not because you have to download 100 GB, but because you need to check every byte, count every hash, run every script, and so on.


    But if you draw the whole process, starting from pressing the "Send" button in your wallet and ending with viewing the block with your transaction somewhere on blockchain.info , then it will be the miners who decide whether your transaction will be in the block or not.


    Sky is the limit?


    nope


    To get started, let's go over the first point again and discuss the concept of money supply.


    One of the fundamental chips that cryptocurrency proponents often flaunt is deflation, which was originally laid down . This is due to the fact that even at the stage of system design, a total limit of 21 million coins (approximately) was indicated, and even if you really want to, raising this threshold will not work. Unlike the ruble or dollar, which at the request of the treasury can be printed in any quantity, which sometimes leads to sad consequences, as in Zimbabwe .


    BTW not everyone thinks deflation is such a definite plus .


    Reward


    The next good question is where did the figure of 21 million come from?


    I think you understand that the amount of coins issued at a particular moment in time is equal to the amount of rewards for the blocks created by that moment . A fairly obvious fact, given that there is only one way in which new coins enter the network.


    But the remuneration is not fixed, and moreover, every 210,000 blocks (approximately once every 4 years) it is halved.


    consensus.nSubsidyHalvingInterval = 210000;
    // https://github.com/bitcoin/bitcoin/blob/master/src/chainparams.cpp#L73

    So, for example, when it all started in January 2009, the block reward was 50 BTC. After 210,000 blocks, in November 2012 it fell to 25 BTC, and most recently, on July 9, 2016, it dropped to 12.5 BTC .


    It is easy to calculate the exact number of Satoshi that will be produced, assuming that Bitcoin does not stall in the next 200 years:


    start_block_reward = 50
    reward_interval = 210000
    def max_money():
        # 50 BTC = 50 0000 0000 Satoshis
        current_reward = 50 * 10**8
        total = 0
        while current_reward > 0:
            total += reward_interval * current_reward
            current_reward /= 2
        return total
    print "Total BTC to ever be created:", max_money(), "Satoshis"
    # Total BTC to ever be created: 2099999997690000 Satoshis

    The picture below shows the production curve, which will increasingly smoothly approach the mark of 21 million BTC, reaching a peak around 2140. At this time, the block reward will become 0 BTC.


    btc_curve


    One can only guess what will happen to Bitcoin then, but one thing we can know for sure - miners will not be left without money. At least they still have a transaction fee , another thing is that this very commission can increase by an order of magnitude.


    Take for clarity some fresh block, for example # 447119 . The amount of commissions from all transactions in the block is approximately 0.78 BTC, while the reward for it is 12.5 BTC. That is, if reward disappears tomorrow , then in our case the commission should grow more than 16 times in order to level this unpleasant event. It is clear that it no longer smells of any micropayments.


    Mining for dummies


    Let's try once again to introduce the mining process at our, so far, primitive level.


    There is a network with a bunch of participants. Some of the participants call themselves miners - they are ready to collect new transactions on their PC, check them for validity, then somehow mine a new block from them, scatter this block over the network and receive money for it. The logical question is - if everything is so simple, then why is every member of the network not doing this?

    It is clear that if everything would be as I described now, then the blocks would go out a hundred times per second, there would be so much currency that no one would give a cent for it, and so on.


    Therefore, Satoshi was forced to come up with an algorithm with the following properties:


    • Creating a new block is a computationally difficult task. You can’t just turn on a powerful PC and mine a hundred blocks in a minute.
    • The entire network takes 10 minutes to calculate a new block (on average). If you look at Litecoin, then there the blocks come out every 2-3 minutes, the bottom line is that the average time is set in advance.
    • Moreover, this time does not depend on the number of network participants. Even if one day there will be a hundred times more miners, the algorithm must change its parameters in such a way that it becomes harder to find the block , and block time drops back to the vicinity of ten minutes.
    • Remember that the network is distributed and peer-to-peer, which means that it must itself understand at what point and how to tighten up these parameters. No control nodes, everything is completely autonomous.
    • If the solution to the task of creating a new block is a complex task that requires time and resources, then checking the block for "correctness" should be simple and almost instant.

    Proof-of-Work (PoW)


    Most likely you are now arriving in complete prostration and do not really understand how this is even possible. But Satoshi was not at a loss and was able to come up with a solution for all these problems - the algorithm was called Proof-of-Work , this is how it looks (I advise you to first read Bitcoin in a nutshell - Blockchain ):


    Let you be a miner. You have 10 transactions that you want to mine into a block. You check these transactions for validity, form a block out of them, specify 0 in the nonce field and consider the block hash. Then change nonce to 1, count the hash again. And so on to infinity.

    Your task is to find a nonce such that a block hash (256 bit number) is less than a predetermined number N. A search for such a hash is possible only by bluntly enumerating nonce, there are no beautiful algorithms here. Therefore, the faster you want to find nonce , the more power you will need.

    The number N is exactly that parameter (it is also called target), which the network sets up depending on the total power of the miners. If tomorrow the blocks begin to come out, relatively speaking, every three minutes, then N will be somehow reduced, it will take longer to search for nonce and the block time will increase again to 10 minutes. And vice versa.

    Technical side


    General view of the algorithm


    It's time to move from words to deeds and demonstrate how Proof-of-Work and mining in general work. And in my humble opinion, there is nothing better than to show the whole process right in combat. To do this, we’ll write our own mining node right away and even try to make a new block before anyone else, although the chances of success are slim.


    Receive transactions


    In a good way, here you need to again dive into the protocol specification, establish contact with other nodes and wait for fresh transactions to be sent to us. In this case, we get a real real-time miner, no worse than ready-made solutions (but this is not accurate).


    I suggest going the simplified way. Open blockchain.info and select several transactions from the Recent Transactions list . They just got into the network and so far are not included in any of the blocks. Next, open another block explorer - chainquery.com . He knows how to issue transactions in raw format and by hashes we get transactions in the form we already know. I limited myself to two ( one , two ):


    txn_pool = []
    txn_pool.append("0100000001440d795fa6267cbae00ae18e921a7b287eaa37d7f41b96ccbc61ef9a323a003d010000006a47304402204137ef9ca79bcd8a953c0def89578838bbe882fe7814d6a7144eaa25ed156f66022043a4ab91a7ee3bf58155d08e5f3f221a783f645daf9ac54fed519e18ca434aea012102965a03e05b2e2983c031b870c9f4afef1141bf30dc5bb993197ee4a52f1443e0feffffff0200a3e111000000001976a914f1cfa585d096ea3c759940d7bacd8c7259bbd4d488ac4e513208000000001976a9146701f2540186d4135eec14dad6cb25bf757fc43088accbd50600")
    txn_pool.append("0100000001517063b3d932693635999b8daaed9ebf020c66c43abf504f3043850bca5a936d010000006a47304402207473cda71b68a414a53e01dc340615958d0d79dd67196c4193a0ebcf0d9f70530220387934e7317b60297f5c6e0ca4bf527faaad830aff45f1f5522e842595939e460121031d53a2c228aedcde79b6ccd2e8f5bcfb56e2046b4681c4ea2173e3c3d7ffc686ffffffff0220bcbe00000000001976a9148cc3704cbb6af566598fea13a3352b46f859581188acba2cfb09000000001976a914b59b9df3700adae0ea819738c89db3c2af4e47d188ac00000000")

    Check


    The next step is to verify the received transactions. I will not do this, just list the main points:


    • Correctly followed transaction structure and syntax
    • I / O list cannot be empty
    • Input transactions must exist either in the UTXO pool or in an unconfirmed transaction pool
    • The sum of the inputs is not less than the sum of the outputs
    • Full list can be found here.

    Some miners reject transactions with zero or too little commission, but everyone decides for himself.


    Sort


    Just in case, I ’ll explain that nothing prevents you from including transactions in the block in any order, the main thing is that they pass all the checks. In my case, there are only two transactions, so sorting them all the more makes no sense. But do not forget that the block size is limited to 1 MB, so if you have 10,000 transactions in the pool, it would be reasonable to sort them by commission and write only the most expensive ones to the block.


    BTW Often there are articles / books that say that before mining a new block, Bitcoin Core sorts transactions by a special priority parameter , which is considered as


    Priority = Sum (Value of input * Input Age) / Transaction Size

    This was true up to version 0.12.0, then sorting by priority was disabled .


    Get reward


    block


    If you look at the structure of any block, the very first always goes the so-called coinbase transaction - it is she who sends the reward to the address of the miner. Unlike regular transactions, coinbase transaction does not spend exits from the UTXO pool as inputs . Instead, it has only one input, called coinbase , which "creates" coins from nothing. There is only one way out of such a transaction. He sends a reward to the miner for the block plus the amount of commissions from all transactions in the block. In my case it is 12.5 + 0.00018939 + 0.0001469 = 12.50033629.


    Let's take a closer look at the structure of a coinbase transaction, and more specifically, its input. Just in case, I’ll remind you what the input of a “normal” transaction looks like:



    Here are three differences in entering a coinbase transaction:


    • Instead of a real transaction hash , 32 null bytes are specified
    • Instead, output index is specified 0xFFFFFFFF.
    • In the unlocking script field, you can specify anything from 2 to 100 bytes in size, so this field is also called coinbase data . For example, in the genesis block, a phrase is hidden there "The Times 03/Jan/2009 Chancellor on brink of second bailout for banks". As a rule, miners insert their name / name of the mining pool / something else into coinbase data .

    Often, so-called extra nonce is inserted into coinbase data , more details here . The bottom line is that you may not find the desired nonce , in which the hash of the block is less than target (in fact, this will happen in most cases). Then it remains to change something in the transaction to get other hashes, for example, UNIX timestamp . But if you read Bitcoin in a nutshell - Blockchain , then you know that you can’t change the timestamp too, otherwise other nodes will reject your block. The solution turned out to be quite simple: just add some number to coinbase data and change it if for the current headernot found needed nonce .


    The process of creating a new transaction is described in detail in the chapter Bitcoin in a nutshell - Protocol , so here I just give the already obtained coinbase transaction , all the code, as usual, is available on [Github] ():


    coinbase_txn = "01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff8a47304402207e8495986ec27ed4556fee9dcd897ea028d4eb2023959c2299eb573e0771dee702201489e40115ccc45d4c23f1109cb56b513543517f3efc0031965ad94d94d3d2d901410497e922cac2c9065a0cac998c0735d9995ff42fb6641d29300e8c0071277eb5b4e770fcc086f322339bdefef4d5b51a23d88755969d28e965dacaaa5d0d2a0e09ffffffff01ddff814a000000001976a91478e10cf8e4bd38266d8fd4ed5c8b430d30a3cde888ac00000000"

    It remains only to calculate merkle root for these three transactions . To do this, use a code fragment from Bitcoin in a nutshell - Blockchain :


    txn_pool.insert(0, coinbase_txn)
    txn_hashes = map(getTxnHash, txn_pool)
    print "Merkle root: ", merkle(txn_hashes)
    # Merkle root:  4b9ff9ab901df82050f858accde99b9169067acafaeade25598ea5505fb53836

    Target


    As I wrote above, all mining comes down to finding a block hash less than a number called target . In the block structure, this number is written in the bits field , for example, for block # 277,316, target was equal 1903a30c.


    $ bitcoin-cli getblock 0000000000000001b6b9a13b095e96db41c4a928b97ef2d944a9b31b2cc7bdc4
    {
        "hash" : "0000000000000001b6b9a13b095e96db41c4a928b97ef2d944a9b31b2cc7bdc4",
        "confirmations" : 35561,
        "size" : 218629,
        "height" : 277316,
        "version" : 2,
        "merkleroot" : "c91c008c26e50763e9f548bb8b2fc323735f73577effbc55502c51eb4cc7cf2e",
        "tx" : ["d5ada064c6417ca25c4308bd158c34b77e1c0eca2a73cda16c737e7424afba2f", 418 more transactions],
        "time" : 1388185914,
        "nonce" : 924591752,
        "bits" : "1903a30c", // Here it's
        "difficulty" : 1180923195.25802612,
        "chainwork" : "000000000000000000000000000000000000000000000934695e92aaf53afa1a",
        "previousblockhash" : "0000000000000002a7bbd25a417c0374cc55261021e8a9ca74442b01284f0569",
        "nextblockhash" : "000000000000000010236c269dd6ed714dd5db39d36b33959079d78dfd431ba7"
    }

    In bits, in fact, two numbers are written at once: the first byte 0x19is the exponent, the remaining three bytes 0x03a30care the mantissa. In order to get the target of BITS, you need to use the following formula: target = mantissa * 2^(8 * exponent - 3)). In the case of block # 277.316, it turns out:


    >>> bits = 0x1903a30c
    >>> exp = bits >> 24
    >>> mant = bits & 0xffffff
    >>> target_hexstr = '%064x' % (mant * (1 << (8 * (exp - 3))))
    >>> target_hexstr
    '0000000000000003a30c00000000000000000000000000000000000000000000'

    Another term reflecting the complexity of mining is difficulty . For example, for block # 449.584, it was equal 392,963,262,344.37. This parameter is a relation max_target / current_target, where max_targetis the maximum possible target, namely 0x00000000FFFF0000000000000000000000000000000000000000000000000000( 0x1d00ffffin bits format). It is bits that are usually specified in all block explorer.


    BTW, the smaller the target, the greater the difficulty and vice versa.


    PoW


    Now that you’ve figured out all the nuances, you can run the miner:


    import hashlib
    import struct
    import sys
    # ======= Header =======
    ver = 2
    prev_block = "000000000000000000e5fb3654e0ae9a2b7d7390e37ee0a7c818ca09fde435f0"
    merkle_root = "6f3ef687979a1f4866cd8842dcbcebd2e47171e54d1cc76c540faecafe133c39"
    bits = 0x10004379 # Not the actual bits, I don't have synced blockchain
    timestamp = 0x58777e25
    # Calculate current time with this code:
    # hex(int(time.mktime(time.strptime('2017-01-12 13:01:25', '%Y-%m-%d %H:%M:%S'))) - time.timezone)
    exp = bits >> 24
    mant = bits & 0xffffff
    target_hexstr = '%064x' % (mant * (1 << (8 * (exp - 3))))
    # '0000000000000000000000000000000000437900000000000000000000000000'
    target_str = target_hexstr.decode('hex')
    # ======== Header =========
    nonce = 0
    while nonce < 0x100000000: # 2**32
        header = ( struct.pack("<L", ver) + prev_block.decode('hex')[::-1] +
              merkle_root.decode('hex')[::-1] + struct.pack("<LLL", timestamp, bits, nonce))
        hash = hashlib.sha256(hashlib.sha256(header).digest()).digest()
        sys.stdout.write("\rNonce: {}, hash: {}".format(nonce, hash[::-1].encode('hex')))
        sys.stdout.flush()
        if hash[::-1] < target_str:
            print 'Success!'
            break
        nonce += 1

    Hash rate


    If you have waited for the coveted line Success!, then you have either Intel Core i7, or a lot of free time. I have no idea when this code will finish its work and whether it will find nonce at all, because the current complexity is simply monstrously great. Even if we assume that our program is able to calculate 100,000 hashes per second (which is not the case), then it is still millions of times slower than any real miner, so it can take several days to search for nonce .


    To make you aware of the extent of the problem: there is a hashrate metric . It reflects the total power of miners in the Bitcoin network, the unit of measure is SHA256 hashes per second. Here is her schedule :


    hashrate


    We assume that the hash rate is 2,000 PH / s = 2,000,000 TH / s = 2,000,000,000 GH / s = 2,000,000,000,000 MH / s = 2,000,000,000,000,000 KH / s. And our program cannot even master 100 KH / s, so there is no point in competing with the entire network.


    2 Blocks 1 Chain


    Fork


    Let's imagine for a moment that miners are looking for block # 123456. And at about the same time, he was found by two independent miners, one of which lives in Australia and the other in the United States. Each of them begins to scatter its version of the block over the network, and as a result it turns out that one half of the world has one blockchain, and the other has another.


    Is this possible and what will happen in this case?


    fork1


    Yes it is possible. Moreover, this happens quite often. In this case, each node continues to adhere to its own version of the blockchain until someone finds the next block. Suppose that the new block continues the green branch, as in the picture below.


    fork2


    In this case, those nodes that adhere to the "red" version automatically synchronize the green one, because the rule works in the Bitcoin world: the longest version of the blockchain is "true" . The "red" version of the blockchain will simply be forgotten, along with rewards for those who find it.


    You may ask: what if the fork goes further? That is, simultaneously with the “purple” block they will find another one that will continue the “red” version of the blockchain?


    Most likely, this book will be read not only by people with a good mathematical education, so I will give the most general answer - this is certainly possible. But the probability of even the first fork is rather small, the second one is even less and so on. To make you understand, the longest fork in the history of Bitcoin was only 4 blocks . So at some point, one of the branches will nevertheless break ahead, and the entire network will go to it.


    If you are interested in this problem from the perspective of probability theory, then you can read "What is the probability of forking in blockchain?" This question is also well described in the famous "Bitcoin: A Peer-to-Peer Electronic Cash System" by Satoshi Nakamoto .


    51% attack


    On the simple fact that the longest chain in the blockchain is dominant, an entire attack is based:


    Imagine that you are a scammer and buy goods at 1000 BTC in a store. You agree with the seller and send him the money. The seller checks the blockchain, sees that such a transaction really was, passed all the checks and even got into some block, for example # 123. After that, the seller goes to the mail and sends you the goods.

    At this time, you turn on your mining farm and start mining, starting from block # 122. If you have enough power, then you can overtake the rest of the network and count the fastest to block # 124, after which the whole world will switch to your version of the blockchain. At the same time, you will not include your transaction for 1000 BTC in any of the blocks, which means that it will be forever forgotten, as if it had never been. As a result, the seller will lose the goods and will not receive his money.

    I will not go into probability theory, but it is impossible to carry out such an attack, unless you have at least half the hashrate of the entire network. You can read more in bitcoin.pdf .


    Nevertheless, some mining pools have very significant capacities. For example, the BTC Guild in 2013 almost crossed the threshold of 51% hashrate. At some point, they immediately mined 6 blocks in a row, so that if they wanted to, they could carry out this attack. Therefore, it is recommended to consider the transaction confirmed only after 6 blocks have been created from above.


    Hardware


    You can immediately forget about mining on a CPU or GPU. For your understanding, the hashrate at the beginning of 2017 is shown below . We assume that it averages 2.300.000 TH / S, i.e. 2.300.000.000.000 MH / s. For comparison, the most savage graphics card such as ATI Radeon HD 5870 Eyefinity or AMD Radeon HD 7970 (x3) , issue at best 2000 MH / S . Among the processors, the first place is occupied by the Xeon Phi 5100 with a funny 140 MH / s.


    hashrate


    So even based on the rate of $ 1000 / BTC and having 10.000 MH / s on hand, you will earn an average of 20 cents per month .


    gpu


    CPU mining ceased to be profitable back in 2011, the GPU held on until about 2013, but also burned out when the so-called application-specific integrated circuit - ASIC became widespread . These are special chips sharpened for mining at the iron level. The simplest ones cost around $ 100, which is much cheaper than a top-end video card, but at the same time they can produce from 1 TH / s.


    That is, ceteris paribus, having two Antminer S9s at $ 3.000 apiece, you will earn almost $ 700 per month (excluding electricity bills)



    But that's not all. You can team up with other miners in the mining pool and start mining together, and divide the money earned in proportion to the invested capacities. This, obviously, is much more profitable than trying to earn at least something alone, which is why pools today are the main driving force in the mining world. At the beginning of 2017, the main players in the pool market are AntPool , F2Pool and Bitfury , which provide more than 40% hashrate of the entire network.


    Pools


    Conclusion


    On this high note, I end my story about the technical device of Bitcoin. Sources of the text plus code examples here , in the same pdf version. Pull requests welcome, ask your questions in Issues or in the comments.


    Obama out