January 30, 2015 at 22:58Why companies do not know how to handle money
It so happened that my career - first as a developer, then as an architect, technical director and finally as an entrepreneur, developed mainly in the near-financial sector. Most of my professional activity was in the design and writing of software for banks, brokers and similar companies. On one of the most typical technical problems of the financial sector, I would like to tell respected Habrachians.
In the public mind, the image of a banker, trader or broker is formed very enviably. This man is the master of life in an expensive suit, dissecting in a brilliant powerful car, unshakably confident in himself and possessing some mystical economic knowledge. He who knows exactly what and how to do so that the money he doesn’t peck at, has turned into an even bigger pile of money.
With the help of its trading terminals, BI systems, risk management systems and other frighteningly cunning software, this embodiment of the archetype of a businessman can do fantastic things - for example, predict the future of markets, making money on the movements of quotes and rates. Perhaps he can calculate his risks with frightening accuracy, guaranteed to remain in the black, or create another financial magic that is inaccessible to mere mortals.
Needless to say, most people are sure that this successful financier knows exactly how much money he has at the moment. And also how much they owe him, and to whom and how much he owes. At any moment, its reliable software that does not know the failures (after all, these are systems where money sits in almost every variable!) Will provide him with this information in all details and sections. On first request, it will give out all the necessary numbers and draw beautiful graphs in real time.
Those of you who worked in financial and similar companies, and dealt with the kernel code (Core Banking, OLTP or their counterpart that implements financial products) - they probably already grin unkindly, remembering something of their own. Because the reality is infinitely far from the picture that arises in the imagination of the average man when he hears the phrase “financial company”.
The sad truth is that companies run by these sharks in expensive suits can only very roughly find out how much money they have at the moment and what’s happening with that money (and therefore with the company) right now.
Confusion in reports, errors in customer accounts, system failures, inconsistent balance in accounting and nervous curses of developers trying to make the code of this whole economy often hide behind a facade of gloss, solidity and professional expression. To understand why this happens, consider a simplified business process and its implementation in most financial institutions.
A typical process in a typical financial company proceeds approximately as follows: Clients interact with the online system by changing the balance values on its account (depositing and withdrawing funds, as well as using financial products that change the state of the account). These operations are performed by a system scientifically called OLTP - Online Transaction Processing, which is designed to perform certain operations with accounts on the fly, thus realizing the company's business.
In the vast majority of quite complex and loaded systems, OLTP can neither provide data for the reporting system, nor even answer the question “how much money is currently in the system and how it is distributed” until the EOD is generated. EOD - End Of Day, a set of procedures that serve to close the accounting day and obtain financial results on it, as well as to calculate the financial products of the company. For example, in most banks, it is during the EOD that interest rates on customer deposits are calculated. EOD is carried out in the system, as a rule, during non-working hours - at night. In this regard, many Internet banks, depending on technical advancement, are partially or completely inaccessible at night.
Thus, with the standard and universally accepted approach to the design of financial systems, the profit for the day can be estimated only after the EOD and all its procedures. In addition, to obtain results for a closed day, living people are often involved - to make corrections, enter coefficients - in short, for everything that has not yet been automated or has the character of “temporary nuances.” For example, if you need to consider not only the results of business logic inside their own system, but also the state of the company’s accounts with contractors who are not connected to the system through the API.
The result of this whole process is that accountants spend a lot of time to keep a balance that does not converge, the profit of an international financial company, taking into account all the amendments, is often considered excel, the reports generated by the system show figures that are more indicative. The numbers obtained in their system, as a rule, agree with the results of counterparties only approximately, with a certain error - this situation is considered normal. It’s bad when they don’t agree at all, which also happens - in such cases it is customary to negotiate, rather than clarify the calculation method. Here we turn to the main thing, namely, to consider one of the reasons why this happens.
Imagine that we have a fairly experienced and professional programmer (or development team) who has been given the task of implementing a financial product. For example, an urgent contribution - i.e. the code should ensure the calculation of interest on the deposit that meets the conditions of the agreement concluded between the client and the bank. The programmer receives the terms and conditions of the contract in the form of a technical task. The developer (or analyst) draws up a model based on the text of the task (at best, maybe he will simply write some valuable instructions for the programmer in the text) and proceeds to implement it in the code. The code, greatly simplifying, ultimately boils down to replacing a certain quantity in the database with a new, calculated quantity. Something like
The value of the account_value field is replaced with a new one, obtained as a result of the work of the created business logic. This means that the client will see the correct state of his account in his Internet bank. And this, quite logical and working implementation, opens the door to hell.
Consider this line more closely in terms of the concept and philosophy of the whole system. We put money on the client’s account, which came from nowhere - from the formula and model, from the resulting business logic code. Which, as we understand it, is not an adequate reflection of the real world - in the Newtonian space objects do not appear from nowhere and do not disappear anywhere. They only move in space, and also change their shape and state.
As practice shows, gradually, with the increasing complexity of the system and its functionality, the amount of code and integration with other systems, fragmented operations with the appearance of money from nowhere and disappearing into nowhere will inevitably lead to errors, failures in business logic and, as a result, to catastrophic financial and reputational losses of the company.
In addition to these dangers, the numbers exchanged by the most diverse and disparate blocks of business logic require long EOD procedures, if only to simply collect the received numbers in a heap and try to issue at least some kind of report on the basis of them. That is why the business is waiting for IT reports for yesterday to understand what was happening with the company’s business at least a day ago.
Meanwhile, the problem of money confusion and misunderstanding of the state of one's own business is by no means new to humanity. Venetian merchants faced the same problem during the Renaissance. With the growth of trade volumes and the number of transactions, the old principles of doing business “on the fingers” were no longer valid. As a result, dealers had exactly the same problem as the modern financial sector (yes, these guys in expensive suits) - confusion in numbers and lack of a clear understanding of how much money and in what form they have at the moment.
The problem of merchants was resolved back in 1494, when the Franciscan monk Luca Bartolomes Pacioli published his treatise “Summa de Arithmetica, Geometria, Proportioni et Proportionalita”, or rather, its chapter “De Computis et Scripturis” (On Calculations and Records) which gave clear instructions on how to conduct business in books, as well as “How to get information about his assets and liabilities as a trader without delay”. From this event, the principle of double-entry, which is universally applied and is the basis of all modern accounting, leads its history.
The problem of the IT financial world is that for the most part, developers, and even architects, rarely know about the Franciscan monk and the principles he describes. As a result, we get the systems on which the financial business depends, but at the same time suffering from troubles that were successfully resolved during the Renaissance.
Returning from the times of cultural prosperity today, we can avoid such a sad situation with IT if we comprehend the process of implementing business logic not through changing the values in the selected fields of the database, but as implementing the accounting approach in the business logic. In this case, we enter into OLTP all the attributes present in the double entry, primarily debit and credit. With this approach, absolutely all operations with accounts are realized through accounting entries or, in other words, financial transactions. (It is important not to confuse the financial transactions in question with the term “transaction” in programming. A financial transaction is the transfer of funds or assets between accounts within the system).
Using the principle of double-entry directly in business logic, we sharply increase the reliability and adequacy of our system, because we exclude a situation where money simply appears and disappears as a result of UPDATE. All operations with accounts we carry out transactions, i.e. moving funds in the system. And this means that we sharply brought our model to the real world. In addition to sharply increasing reliability due to a constantly guaranteed converging balance (which means huge savings on the manual work of accountants), we have at our disposal all the power of General Ledger and the accounting approach in working with numbers and reports.
Due to this approach to the implementation of the core business products, reporting is simplified both in terms of development and in terms of load. This paves the way for the implementation of an EOD-free system, which is also capable of issuing reports in real time - which means that the business will no longer be excited to wait for the results of yesterday. Instead, he will have information at the moment, and accordingly - feel much more confident, be able to act much more effectively than before.
The architectural core, in which the double-entry principle is rigidly implemented, can be used to implement almost any financial and near-financial products on its basis - in Core Banking, OLTP, Wealth Management, Treasury, Accounting, Payment Processing, ERP systems and any solutions that operate certain assets. Having such a core in the form of a separate base architectural layer, on which the rest of the functionality is based, we not only greatly increase the security of the entire system, but also achieve significant cost savings while improving quality in the development of new products and services.
Having seen enough of how the companies run around the old rake in circles, my colleagues and I decided to create our own company and started developing such a kernel, as well as projects and systems based on it in accordance with the characteristics of the customer’s business. The described principle is only one of many others that form the basis of our product. If the article arouses interest and positive discussion, I will continue the story about other typical horrors and troubles of the financial sector, as well as the proposed ways to resolve them.
In the public mind, the image of a banker, trader or broker is formed very enviably. This man is the master of life in an expensive suit, dissecting in a brilliant powerful car, unshakably confident in himself and possessing some mystical economic knowledge. He who knows exactly what and how to do so that the money he doesn’t peck at, has turned into an even bigger pile of money.
With the help of its trading terminals, BI systems, risk management systems and other frighteningly cunning software, this embodiment of the archetype of a businessman can do fantastic things - for example, predict the future of markets, making money on the movements of quotes and rates. Perhaps he can calculate his risks with frightening accuracy, guaranteed to remain in the black, or create another financial magic that is inaccessible to mere mortals.
Needless to say, most people are sure that this successful financier knows exactly how much money he has at the moment. And also how much they owe him, and to whom and how much he owes. At any moment, its reliable software that does not know the failures (after all, these are systems where money sits in almost every variable!) Will provide him with this information in all details and sections. On first request, it will give out all the necessary numbers and draw beautiful graphs in real time.
Those of you who worked in financial and similar companies, and dealt with the kernel code (Core Banking, OLTP or their counterpart that implements financial products) - they probably already grin unkindly, remembering something of their own. Because the reality is infinitely far from the picture that arises in the imagination of the average man when he hears the phrase “financial company”.
The sad truth is that companies run by these sharks in expensive suits can only very roughly find out how much money they have at the moment and what’s happening with that money (and therefore with the company) right now.
Confusion in reports, errors in customer accounts, system failures, inconsistent balance in accounting and nervous curses of developers trying to make the code of this whole economy often hide behind a facade of gloss, solidity and professional expression. To understand why this happens, consider a simplified business process and its implementation in most financial institutions.
A typical process in a typical financial company proceeds approximately as follows: Clients interact with the online system by changing the balance values on its account (depositing and withdrawing funds, as well as using financial products that change the state of the account). These operations are performed by a system scientifically called OLTP - Online Transaction Processing, which is designed to perform certain operations with accounts on the fly, thus realizing the company's business.
In the vast majority of quite complex and loaded systems, OLTP can neither provide data for the reporting system, nor even answer the question “how much money is currently in the system and how it is distributed” until the EOD is generated. EOD - End Of Day, a set of procedures that serve to close the accounting day and obtain financial results on it, as well as to calculate the financial products of the company. For example, in most banks, it is during the EOD that interest rates on customer deposits are calculated. EOD is carried out in the system, as a rule, during non-working hours - at night. In this regard, many Internet banks, depending on technical advancement, are partially or completely inaccessible at night.
Thus, with the standard and universally accepted approach to the design of financial systems, the profit for the day can be estimated only after the EOD and all its procedures. In addition, to obtain results for a closed day, living people are often involved - to make corrections, enter coefficients - in short, for everything that has not yet been automated or has the character of “temporary nuances.” For example, if you need to consider not only the results of business logic inside their own system, but also the state of the company’s accounts with contractors who are not connected to the system through the API.
The result of this whole process is that accountants spend a lot of time to keep a balance that does not converge, the profit of an international financial company, taking into account all the amendments, is often considered excel, the reports generated by the system show figures that are more indicative. The numbers obtained in their system, as a rule, agree with the results of counterparties only approximately, with a certain error - this situation is considered normal. It’s bad when they don’t agree at all, which also happens - in such cases it is customary to negotiate, rather than clarify the calculation method. Here we turn to the main thing, namely, to consider one of the reasons why this happens.
Imagine that we have a fairly experienced and professional programmer (or development team) who has been given the task of implementing a financial product. For example, an urgent contribution - i.e. the code should ensure the calculation of interest on the deposit that meets the conditions of the agreement concluded between the client and the bank. The programmer receives the terms and conditions of the contract in the form of a technical task. The developer (or analyst) draws up a model based on the text of the task (at best, maybe he will simply write some valuable instructions for the programmer in the text) and proceeds to implement it in the code. The code, greatly simplifying, ultimately boils down to replacing a certain quantity in the database with a new, calculated quantity. Something like
UPDATE accounts SET account_value = v_new_account_value WHERE customer_id = v_customer_id;
The value of the account_value field is replaced with a new one, obtained as a result of the work of the created business logic. This means that the client will see the correct state of his account in his Internet bank. And this, quite logical and working implementation, opens the door to hell.
Consider this line more closely in terms of the concept and philosophy of the whole system. We put money on the client’s account, which came from nowhere - from the formula and model, from the resulting business logic code. Which, as we understand it, is not an adequate reflection of the real world - in the Newtonian space objects do not appear from nowhere and do not disappear anywhere. They only move in space, and also change their shape and state.
As practice shows, gradually, with the increasing complexity of the system and its functionality, the amount of code and integration with other systems, fragmented operations with the appearance of money from nowhere and disappearing into nowhere will inevitably lead to errors, failures in business logic and, as a result, to catastrophic financial and reputational losses of the company.
In addition to these dangers, the numbers exchanged by the most diverse and disparate blocks of business logic require long EOD procedures, if only to simply collect the received numbers in a heap and try to issue at least some kind of report on the basis of them. That is why the business is waiting for IT reports for yesterday to understand what was happening with the company’s business at least a day ago.
Meanwhile, the problem of money confusion and misunderstanding of the state of one's own business is by no means new to humanity. Venetian merchants faced the same problem during the Renaissance. With the growth of trade volumes and the number of transactions, the old principles of doing business “on the fingers” were no longer valid. As a result, dealers had exactly the same problem as the modern financial sector (yes, these guys in expensive suits) - confusion in numbers and lack of a clear understanding of how much money and in what form they have at the moment.
The problem of merchants was resolved back in 1494, when the Franciscan monk Luca Bartolomes Pacioli published his treatise “Summa de Arithmetica, Geometria, Proportioni et Proportionalita”, or rather, its chapter “De Computis et Scripturis” (On Calculations and Records) which gave clear instructions on how to conduct business in books, as well as “How to get information about his assets and liabilities as a trader without delay”. From this event, the principle of double-entry, which is universally applied and is the basis of all modern accounting, leads its history.
The problem of the IT financial world is that for the most part, developers, and even architects, rarely know about the Franciscan monk and the principles he describes. As a result, we get the systems on which the financial business depends, but at the same time suffering from troubles that were successfully resolved during the Renaissance.
Returning from the times of cultural prosperity today, we can avoid such a sad situation with IT if we comprehend the process of implementing business logic not through changing the values in the selected fields of the database, but as implementing the accounting approach in the business logic. In this case, we enter into OLTP all the attributes present in the double entry, primarily debit and credit. With this approach, absolutely all operations with accounts are realized through accounting entries or, in other words, financial transactions. (It is important not to confuse the financial transactions in question with the term “transaction” in programming. A financial transaction is the transfer of funds or assets between accounts within the system).
Using the principle of double-entry directly in business logic, we sharply increase the reliability and adequacy of our system, because we exclude a situation where money simply appears and disappears as a result of UPDATE. All operations with accounts we carry out transactions, i.e. moving funds in the system. And this means that we sharply brought our model to the real world. In addition to sharply increasing reliability due to a constantly guaranteed converging balance (which means huge savings on the manual work of accountants), we have at our disposal all the power of General Ledger and the accounting approach in working with numbers and reports.
Due to this approach to the implementation of the core business products, reporting is simplified both in terms of development and in terms of load. This paves the way for the implementation of an EOD-free system, which is also capable of issuing reports in real time - which means that the business will no longer be excited to wait for the results of yesterday. Instead, he will have information at the moment, and accordingly - feel much more confident, be able to act much more effectively than before.
The architectural core, in which the double-entry principle is rigidly implemented, can be used to implement almost any financial and near-financial products on its basis - in Core Banking, OLTP, Wealth Management, Treasury, Accounting, Payment Processing, ERP systems and any solutions that operate certain assets. Having such a core in the form of a separate base architectural layer, on which the rest of the functionality is based, we not only greatly increase the security of the entire system, but also achieve significant cost savings while improving quality in the development of new products and services.
Having seen enough of how the companies run around the old rake in circles, my colleagues and I decided to create our own company and started developing such a kernel, as well as projects and systems based on it in accordance with the characteristics of the customer’s business. The described principle is only one of many others that form the basis of our product. If the article arouses interest and positive discussion, I will continue the story about other typical horrors and troubles of the financial sector, as well as the proposed ways to resolve them.