Data Validation Essay
In the note “Is it possible to divide by 0.01?” I wrote on the testers website that during testing, you need to check the consistency of the input data validators with the application processing logic. But from the comments on this note, I realized that in order to understand how to test data validation, you need to understand how it should work, what can be considered correct and what is not. Therefore, I wrote a separate article about this. It discusses three questions: 1) why is data validation needed at all, and 2) where and when can data validation be performed, 3) what types of checks are possible. And of course, it’s demonstrated how all this looks on living examples. Or maybe my reasoning will be interesting not only to testers, but also to developers.
It would seem that "invalid" data that does not meet certain restrictions can cause a program to crash. But what does this mean? Suppose an exception occurs at some point in the program when trying to convert a string to a number if the string has an incorrect format. Of course, if the exception is not caught anywhere, this can lead to an abnormal termination of the program. But this is an unlikely scenario. Most likely, in some place an interceptor will work, which will either give the user some kind of error message in the program, or write to the error log, after which the program will try to recover from the failure and continue working. That is, even if the validation is not performed, it is likely that nothing bad will happen.
But there can still be some negative consequences for the lack of validation, let's take a closer look at what problems may arise.
In short, the lack of validation can lead to the above (and maybe even some other) problems. Accordingly, the presence of validation helps prevent serious failures, simplifies the identification of problems, but you have to pay for it with performance, since additional checks increase the load on the system. And here we turn to the second question - how to reduce this additional load.
As mentioned above, from the point of view of reducing the load, it is best not to validate the data at all.
But if you still need verification, the logic suggests that it is convenient to check the data in the place where it enters the program from the outside world. After such a check, you can be sure that the correct data gets into the program and that in the future they can be used without additional checks. This can be a user interface through which a person enters data. This can be a file containing program settings or data that the program should process. It can be a database into which information can get from other programs. It may be a network protocol for exchanging data with other programs. Finally, it can be a program interface that another program uses, calling some functions / procedures and passing parameters to them.
Alas, common sense is sometimes forced to retreat before the onslaught of reality. “Face control” of the input data is sometimes not just impractical, but not possible at all. Below are some reasons for this.
However, wherever validation is performed, you can do this in several different ways, depending on what restrictions are placed on the data.
A transaction is a sequence of actions that either all complete successfully, or some kind of failure occurs when a separate action is performed, and then the results of all previous actions of this chain are canceled. So, validation can be performed during the transaction, and the last check can be performed at the very end of the data processing transaction. At the same time, we validate not the data itself, but the state that turned out after they were completely processed, and if this state does not satisfy some restrictions, then we recognize the input data as invalid and return everything to its original state.
What method of validation should be applied in practice in this or that case? Most often, one way to limit oneself is not possible, and it is not necessary. Validation of data can and should be performed in several stages, complicating checks.
First, as you type, make sure that the data does not contain invalid characters. For example, for a numeric field, the user may not be allowed to enter non-digital characters.
After the input is completed, you can check the whole value. There may be some restrictions for the entered number, for example, it should not exceed a certain maximum permissible value. If our number field is age, it should be between 0 and, say, 120.
When all the fields are filled in, you can check whether the entered values are consistent with each other. For example, if in addition to the age field, there is a field in the form for entering the passport number, the application can verify that when filling out the passport number, the age must be at least 14 years.
Finally, if everything is entered correctly, you can try to start processing by performing checks along the way, as well as at the very end, and if something went wrong, roll back to the original state.
Well, of course, checks at the next level can hedge checks at previous levels. Say, for web applications, it is mandatory to check the data that came to the server in an HTTP request, regardless of whether preliminary validation was performed in the browser or not. The reason for this is that client-side validation can be circumvented. For other types of applications, bypassing checks is not so simple, but sometimes it is also quite possible, as shown in the example below.
We conclude the article with a demonstration of various types of validators, as well as some recommendations on how to verify the correctness of their work during testing.
Let's start with a character-by-character check. Graphic editor Paint, dialog for resizing a picture, width of a picture. Only numbers can be entered in this field, when trying to enter other characters, an error message is
displayed : However, being smart, you can bypass this validation of the entered characters: you can paste a negative number into the field through the clipboard, despite the fact that the minus is an invalid character :
However, this does not lead to negative consequences, because at the next level there is another check that works when you click OK:
There are other restrictions for this field, which are also checked after clicking the OK button:
But the field for entering the slope of the picture , which is located very close in the same dialog, does not contain character validation, despite the fact that it is also a numerical field. Moreover, when you enter invalid characters after clicking OK, you can see such a strange message that is practically undecipherable:
All the above examples are associated with checking a single field. An example of validating a combination of fields can be found in the same application, but in a different place - in the page settings dialog for printing. If you specify the size of the page margins so that in total they exceed the page width, we get this message:
Well, finally, in the note"Why is there not enough memory to reduce the size of the picture?" The error described is related to the fact that in this graphic editor there is no correct error handling and transaction rollback if the picture size is too large.
The tester needs to work out all these situations. First, validation needs to be verified at all levels. Secondly, it is necessary to check the consistency of validators at different levels. Thirdly, it is necessary to look for ways to bypass validators, trying to get to the next level without preliminary checks.
Most of this article is not devoted to testing validators, but to describing their structure. Why? Because the enemy must be known in person. To find a data validation defect, you need to understand where to look and what to pay attention to.
PS Crosspost
Why is data validation necessary?
It would seem that "invalid" data that does not meet certain restrictions can cause a program to crash. But what does this mean? Suppose an exception occurs at some point in the program when trying to convert a string to a number if the string has an incorrect format. Of course, if the exception is not caught anywhere, this can lead to an abnormal termination of the program. But this is an unlikely scenario. Most likely, in some place an interceptor will work, which will either give the user some kind of error message in the program, or write to the error log, after which the program will try to recover from the failure and continue working. That is, even if the validation is not performed, it is likely that nothing bad will happen.
But there can still be some negative consequences for the lack of validation, let's take a closer look at what problems may arise.
- Inability to recover from a failure. Not always the program is able to "return everything back." Perhaps, in the process, the program performed some irreversible actions - deleted the file, sent data over the network, printed something to the printer, launched the machine cutter, and it partially processed the workpiece blank. But even if recovery is possible in principle, the recovery algorithm may also contain errors, and this sometimes leads to very sad consequences.
- Additional system load. Failure recovery is an unnecessary job. All the work that was done before the failure was also superfluous. And this means an additional load on the system, which can be avoided if you check the data in advance. On the other hand, validation is also an additional burden, and recovery has to be done only occasionally, and the verification must be performed every time, so it is still unknown what is more profitable.
- Injections do not cause malfunctions. One of the main ways to exploit vulnerabilities in programs is to “trick” the validators, that is, to transfer data that the validator recognizes as correct, but at the same time they are interpreted in an unexpected way, so that an attacker can gain unauthorized access to data or some features of the program, either capable of destroying data or a program. If there is no validation at all, the task of the attacker is simplified as much as possible.
- The difficulty of identifying the cause of the problem. If an exception has flown from somewhere in the depth of the program, determining the causes of its occurrence is not so simple. And even if this is possible, it may not be easy to explain to the user that the failure was caused by data that he entered some time ago in some completely different place in the program. And if the check is performed immediately after entering the data, no difficulties with identifying the source of the problem arise.
In short, the lack of validation can lead to the above (and maybe even some other) problems. Accordingly, the presence of validation helps prevent serious failures, simplifies the identification of problems, but you have to pay for it with performance, since additional checks increase the load on the system. And here we turn to the second question - how to reduce this additional load.
Where and when to perform data validation?
As mentioned above, from the point of view of reducing the load, it is best not to validate the data at all.
But if you still need verification, the logic suggests that it is convenient to check the data in the place where it enters the program from the outside world. After such a check, you can be sure that the correct data gets into the program and that in the future they can be used without additional checks. This can be a user interface through which a person enters data. This can be a file containing program settings or data that the program should process. It can be a database into which information can get from other programs. It may be a network protocol for exchanging data with other programs. Finally, it can be a program interface that another program uses, calling some functions / procedures and passing parameters to them.
Alas, common sense is sometimes forced to retreat before the onslaught of reality. “Face control” of the input data is sometimes not just impractical, but not possible at all. Below are some reasons for this.
- Validation requires access to an inaccessible part of the system state. This is especially true for validating human input through a graphical user interface. Modern applications are often built using a multilevel architecture, which assumes that the user interface implementation is highlighted in the presentation layer, and verification requires access to other layers, up to the database layer.
This is especially noticeable for web applications, where the user interface is implemented in the browser and performed on the client side, and to verify the input, a comparison with what is stored in the database is required. In this situation, the verification has to be performed after sending data to the server. (However, now with the advent of AJAX technology, this problem has been partially resolved). - Validation requires a complete repetition of the processing logic. As already noted in the two paragraphs above, with a multi-layered application architecture, the user interface is usually allocated to a special presentation layer, and the data processing logic is on another layer. And there are situations when, for validation, you need to perform this processing almost completely, because there is no shorter way to understand whether it will succeed or not.
How to perform data validation?
However, wherever validation is performed, you can do this in several different ways, depending on what restrictions are placed on the data.
- Character-by-character verification. Typically, such checks are performed in the user interface as data is entered. But not only. For example, the lexical analyzer of the compiler also detects invalid characters directly in the process of reading the compiled file. Therefore, such checks can be arbitrarily called "lexical".
- Checking individual values. For the user interface, this is a check of the value in a separate field, and it can be performed both as you type (it checks the incomplete value that has been entered so far), and after completion of the input, when the field loses focus. For a program interface (API), this is a check of one of the parameters passed to the called procedure. For data received from a file, this is a check of some read fragment of the file. Such checks, again by analogy with compiler terminology, can be called “syntactic”.
- The set of input values. It can be assumed that some data is first transferred to the program, after which a certain signal is generated that initiates their processing. For example, a user entered data in a form or in several forms (in the so-called “wizard”) and finally clicked the “OK” button. At this point, you can perform the so-called "semantic" checks aimed at validating not only individual values, but also the relationships between them, mutual restrictions.
It is quite possible that each individual value is “syntactically” correct, but together they form an inconsistent set. For the software interface, this type of validation involves checking the set of input parameters of the called procedure, for the case of receiving data from a file, it is a check of all read data. - Checking the status of the system after data processing. Finally, there is the last method that you can resort to if the validation of the input data directly fails - you can try to process them, but leave the opportunity to return everything to its original state. Such a mechanism is often called transactional.
A transaction is a sequence of actions that either all complete successfully, or some kind of failure occurs when a separate action is performed, and then the results of all previous actions of this chain are canceled. So, validation can be performed during the transaction, and the last check can be performed at the very end of the data processing transaction. At the same time, we validate not the data itself, but the state that turned out after they were completely processed, and if this state does not satisfy some restrictions, then we recognize the input data as invalid and return everything to its original state.
What method of validation should be applied in practice in this or that case? Most often, one way to limit oneself is not possible, and it is not necessary. Validation of data can and should be performed in several stages, complicating checks.
First, as you type, make sure that the data does not contain invalid characters. For example, for a numeric field, the user may not be allowed to enter non-digital characters.
After the input is completed, you can check the whole value. There may be some restrictions for the entered number, for example, it should not exceed a certain maximum permissible value. If our number field is age, it should be between 0 and, say, 120.
When all the fields are filled in, you can check whether the entered values are consistent with each other. For example, if in addition to the age field, there is a field in the form for entering the passport number, the application can verify that when filling out the passport number, the age must be at least 14 years.
Finally, if everything is entered correctly, you can try to start processing by performing checks along the way, as well as at the very end, and if something went wrong, roll back to the original state.
Well, of course, checks at the next level can hedge checks at previous levels. Say, for web applications, it is mandatory to check the data that came to the server in an HTTP request, regardless of whether preliminary validation was performed in the browser or not. The reason for this is that client-side validation can be circumvented. For other types of applications, bypassing checks is not so simple, but sometimes it is also quite possible, as shown in the example below.
Validator Testing
We conclude the article with a demonstration of various types of validators, as well as some recommendations on how to verify the correctness of their work during testing.
Let's start with a character-by-character check. Graphic editor Paint, dialog for resizing a picture, width of a picture. Only numbers can be entered in this field, when trying to enter other characters, an error message is
displayed : However, being smart, you can bypass this validation of the entered characters: you can paste a negative number into the field through the clipboard, despite the fact that the minus is an invalid character :
However, this does not lead to negative consequences, because at the next level there is another check that works when you click OK:
There are other restrictions for this field, which are also checked after clicking the OK button:
But the field for entering the slope of the picture , which is located very close in the same dialog, does not contain character validation, despite the fact that it is also a numerical field. Moreover, when you enter invalid characters after clicking OK, you can see such a strange message that is practically undecipherable:
All the above examples are associated with checking a single field. An example of validating a combination of fields can be found in the same application, but in a different place - in the page settings dialog for printing. If you specify the size of the page margins so that in total they exceed the page width, we get this message:
Well, finally, in the note"Why is there not enough memory to reduce the size of the picture?" The error described is related to the fact that in this graphic editor there is no correct error handling and transaction rollback if the picture size is too large.
The tester needs to work out all these situations. First, validation needs to be verified at all levels. Secondly, it is necessary to check the consistency of validators at different levels. Thirdly, it is necessary to look for ways to bypass validators, trying to get to the next level without preliminary checks.
Conclusion
Most of this article is not devoted to testing validators, but to describing their structure. Why? Because the enemy must be known in person. To find a data validation defect, you need to understand where to look and what to pay attention to.
PS Crosspost
