September 14, 2015 at 00:22Writing your own Bitcoin payment gateway
UPDATE Opened payment gateway: github.com/Overtorment/Cashier-BTC
For various reasons, existing payment gateways (such as Bitpay) may not suit you. In this article, we will consider creating your own Bitcoin gateway from scratch.
It is assumed that the reader is familiar with the Bitcoin network device. If not, I recommend these articles: “How the Bitcoin Protocol Actually Works” and “Bitcoin: An Introduction for Developers”
Conditionally, I would divide our proposed system into 4 parts:
To perform these operations, we need to select a set of software solutions that will simplify our lives.
The most popular and used solutions can perform all or almost all of the above operations, but nothing prevents us from combining solutions, or even writing your bike for certain stages.
In general, any cryptographic library supporting elliptical cryptography can handle this.
Ordinary Bitcoin libraries for working with Bitcoin are also suitable:
The most “heavyweight” item.
The classic solution is to raise your own reference complete Bitcoin node, which is also the canonical bitcoind. This will allow us to communicate with him via JSON-RPC. With it, we can both receive information from the network and push transactions.
What you should pay attention to:
An alternative is the implementation of a full node in Ruby + PostgreSQL, Toshi . Non-canonical, but striving for full compatibility implementation. Please note, due to additional indexes, the database will occupy 220+ gigabytes of space. Again, synchronization with the network is required.
Perhaps there are other implementations of the full node (unknown to me).
Another alternative is to use a public provider API. It will be on his shoulders to receive information from the network and broadcast transactions.
Now there are:
Personally, I recommend connecting several solutions with a failover.
Depending on which main library we have chosen, this library can or cannot create and sign transactions.
You can write yourself.
See the “Working with Addresses” section.
The result of creating and signing transactions is binary data (hex), ready to push into the network. Until the network sees the transaction, consider that there is no transaction. When a network sees a transaction, it is considered unconfirmed. It is enough to transfer the transaction to one Bitcoin node, after which most of the Bitcoin network will see the transaction in a matter of seconds.
Some client from the section “Working with addresses” (through some hard-coded endpoints), or any full node can transmit transactions. You can broadcast a transaction even by hand, by going to a special page of the Bitcoin provider’s API and driving the transaction into a special form. Canonically, a confirmed transaction is a transaction included in 6 or more consecutive blocks (or in 1-3. Non-canonical, but faster). Transactions with zero (or insufficient) commission may remain unacknowledged for a long time (up to a month, in my practice). Such transactions are preferably periodically relayed.
Suppose we have a unique invoice (invoice, order) submitted for payment to the client, and the client will pay in bitcoins.
To begin with, you need to convert the original account currency (USD for example) to BTC. The task is trivial and we will not consider it.
Further. The de facto standard is the generation of a new unique Bitcoin address for each order (it’s an account, it’s also an invoice, it’s an order). It is expected that only our client will transfer funds to this account, only 1 time, and only the strictly specified amount (more can be, no one will be offended, but no less). T.O. upon receipt of funds to the specified bitcoin address in the required quantity, the order is considered paid.
In short, the chain is this:
When bitcoins arrive at the address, you have options to count the unconfirmed or confirmed balance.
There is a small chance that the transaction will be rolled back, and this may be due to the fault of the payer (who is actually an attacker), or due to circumstances beyond his control.
If you have the opportunity to “take away” the provided goods or service from the client in the event of the fact that the transaction was canceled, I recommend that you account for the unconfirmed balance. This will mean an almost instant payment process for the client (as opposed to hours of waiting, for example). And if some transactions appear to be rolled back as a result, ask the client for a re-payment, threatening to select the service / product.
Do not expect that such fraud will overtake you immediately, rollback of transactions is very rare, and “manually” stimulating such a rollback (by the way, an attacker has no guarantee of success) is not realistic for technically savvy customers (as opposed to credit card chargebacks).
Another example where you can count an unconfirmed balance is if it takes more than one hour to prepare a customer order (for example, a shopping cart is processed, prepared for delivery by courier). There is a lot of time to double-check the balance before sending the goods.
For other cases, you can enter a certain threshold, above which you must expect a confirmed balance (for example, 0.25 BTC). For maximum reliability, make it zero.
After closing the order, you can leave bitcoins at this address on demand, or for convenience transfer to a single “aggregation” merchant wallet. Be careful, in the latter case, you can compromise such a commercial indicator as “turnover”, because Each paying customer can track a payment transaction.
For transfers, you will need to create, sign and broadcast transactions using private keys from addresses.
A few words about the lifetime of the order.
If your product or service is rigidly tied to the equivalent in fiat currency (for example, USD), then the typical lifetime of an order is 7-15 minutes due to the volatility of the exchange rate.
Suitable when you do not invoice for payment, and the user’s account contains a single balance, which he replenishes and spends.
Here you will need to generate a bitcoin address for the user, and show him, asking to top up for any amount.
In this case, it is necessary to monitor the address of incoming transactions, replenish the user internal balance if there are any.
In this case, I recommend counting only confirmed transactions (from 3 blocks and above).
In case of hacking, the attacker will be interested in only one thing - private keys from the addresses generated by you, because they allow you to transfer funds from these addresses anywhere.
I recommend storing them in isolation from the main system in some secure storage. Ultimately, you will need them only when you yourself want to use the proceeds of bitcoins (spend, convert to cash etc).
If necessary, the obtained bitcoins can be automatically entered into the exchange and sold using the exchange API. To do this, we will need to create, sign and broadcast transactions using private keys from addresses.
That's all, I hope it will be useful to those who have a similar problem.
Corrections of inaccuracies and errors are welcome in PM.
For various reasons, existing payment gateways (such as Bitpay) may not suit you. In this article, we will consider creating your own Bitcoin gateway from scratch.
It is assumed that the reader is familiar with the Bitcoin network device. If not, I recommend these articles: “How the Bitcoin Protocol Actually Works” and “Bitcoin: An Introduction for Developers”
Conditionally, I would divide our proposed system into 4 parts:
- Work with addresses . Generation of a pair of public and private keys (as you know, a hash of a public key is a bitcoin address, and the corresponding private key allows them to manage)
- Getting information from the bitcoin network . Transaction status, address balance.
- Creation and signature of transactions . Formation of the correct transaction, signing with the key / keys, conversion to hex. The resulting hex is ready for broadcast to the network.
- Broadcast Transaction . Aka broadcast, send, push. Transferring the hex transaction of the Bitcoin network so that the miners begin work on including the transaction in the blockchain.
To perform these operations, we need to select a set of software solutions that will simplify our lives.
The most popular and used solutions can perform all or almost all of the above operations, but nothing prevents us from combining solutions, or even writing your bike for certain stages.
Work with addresses
In general, any cryptographic library supporting elliptical cryptography can handle this.
Ordinary Bitcoin libraries for working with Bitcoin are also suitable:
- pybitcointools (Vitalik Buterin) - python
- Bitcore (Bitpay) - javascript
- BitcoinJS - javascript
- BitcoinJ - java
- other
Getting information from the bitcoin network
The most “heavyweight” item.
The classic solution is to raise your own reference complete Bitcoin node, which is also the canonical bitcoind. This will allow us to communicate with him via JSON-RPC. With it, we can both receive information from the network and push transactions.
What you should pay attention to:
- After installation, node synchronization may take a long time. Only after synchronization can the node be used.
- It will take a lot of space. Already 40+ gigabytes .
- I personally do not know what kind of demand load I can handle.
- Any problems with the fall / update will fall on your shoulders.
An alternative is the implementation of a full node in Ruby + PostgreSQL, Toshi . Non-canonical, but striving for full compatibility implementation. Please note, due to additional indexes, the database will occupy 220+ gigabytes of space. Again, synchronization with the network is required.
Perhaps there are other implementations of the full node (unknown to me).
Another alternative is to use a public provider API. It will be on his shoulders to receive information from the network and broadcast transactions.
Now there are:
- chain.com
- blockchain.info/ru/api (not recommended)
- www.blockcypher.com
- chain.so/api
- coinalytics.co
- www.blocktrail.com
- coinkite.com/developers
- other
Personally, I recommend connecting several solutions with a failover.
Creating and signing transactions
Depending on which main library we have chosen, this library can or cannot create and sign transactions.
You can write yourself.
See the “Working with Addresses” section.
Transaction translation
The result of creating and signing transactions is binary data (hex), ready to push into the network. Until the network sees the transaction, consider that there is no transaction. When a network sees a transaction, it is considered unconfirmed. It is enough to transfer the transaction to one Bitcoin node, after which most of the Bitcoin network will see the transaction in a matter of seconds.
Some client from the section “Working with addresses” (through some hard-coded endpoints), or any full node can transmit transactions. You can broadcast a transaction even by hand, by going to a special page of the Bitcoin provider’s API and driving the transaction into a special form. Canonically, a confirmed transaction is a transaction included in 6 or more consecutive blocks (or in 1-3. Non-canonical, but faster). Transactions with zero (or insufficient) commission may remain unacknowledged for a long time (up to a month, in my practice). Such transactions are preferably periodically relayed.
General principles of the payment gateway
Option 1
Suppose we have a unique invoice (invoice, order) submitted for payment to the client, and the client will pay in bitcoins.
To begin with, you need to convert the original account currency (USD for example) to BTC. The task is trivial and we will not consider it.
Further. The de facto standard is the generation of a new unique Bitcoin address for each order (it’s an account, it’s also an invoice, it’s an order). It is expected that only our client will transfer funds to this account, only 1 time, and only the strictly specified amount (more can be, no one will be offended, but no less). T.O. upon receipt of funds to the specified bitcoin address in the required quantity, the order is considered paid.
In short, the chain is this:
- order in the system ->
- generate a unique bitcoin address corresponding to the order ->
- show the client ->
- We are waiting for payment to the address ->
- the order is closed (cancellation after the expiration date or the receipt of BTC and we count the fact of payment)
When bitcoins arrive at the address, you have options to count the unconfirmed or confirmed balance.
There is a small chance that the transaction will be rolled back, and this may be due to the fault of the payer (who is actually an attacker), or due to circumstances beyond his control.
If you have the opportunity to “take away” the provided goods or service from the client in the event of the fact that the transaction was canceled, I recommend that you account for the unconfirmed balance. This will mean an almost instant payment process for the client (as opposed to hours of waiting, for example). And if some transactions appear to be rolled back as a result, ask the client for a re-payment, threatening to select the service / product.
Do not expect that such fraud will overtake you immediately, rollback of transactions is very rare, and “manually” stimulating such a rollback (by the way, an attacker has no guarantee of success) is not realistic for technically savvy customers (as opposed to credit card chargebacks).
Another example where you can count an unconfirmed balance is if it takes more than one hour to prepare a customer order (for example, a shopping cart is processed, prepared for delivery by courier). There is a lot of time to double-check the balance before sending the goods.
For other cases, you can enter a certain threshold, above which you must expect a confirmed balance (for example, 0.25 BTC). For maximum reliability, make it zero.
After closing the order, you can leave bitcoins at this address on demand, or for convenience transfer to a single “aggregation” merchant wallet. Be careful, in the latter case, you can compromise such a commercial indicator as “turnover”, because Each paying customer can track a payment transaction.
For transfers, you will need to create, sign and broadcast transactions using private keys from addresses.
A few words about the lifetime of the order.
If your product or service is rigidly tied to the equivalent in fiat currency (for example, USD), then the typical lifetime of an order is 7-15 minutes due to the volatility of the exchange rate.
Option 2
Suitable when you do not invoice for payment, and the user’s account contains a single balance, which he replenishes and spends.
Here you will need to generate a bitcoin address for the user, and show him, asking to top up for any amount.
In this case, it is necessary to monitor the address of incoming transactions, replenish the user internal balance if there are any.
In this case, I recommend counting only confirmed transactions (from 3 blocks and above).
- generation of the address to the user ->
- monitoring transactions to an address ->
- replenishment of the internal account in the presence of incoming transactions
A few words about security
In case of hacking, the attacker will be interested in only one thing - private keys from the addresses generated by you, because they allow you to transfer funds from these addresses anywhere.
I recommend storing them in isolation from the main system in some secure storage. Ultimately, you will need them only when you yourself want to use the proceeds of bitcoins (spend, convert to cash etc).
What's next?
If necessary, the obtained bitcoins can be automatically entered into the exchange and sold using the exchange API. To do this, we will need to create, sign and broadcast transactions using private keys from addresses.
That's all, I hope it will be useful to those who have a similar problem.
Corrections of inaccuracies and errors are welcome in PM.