October 22, 2009 at 11:00About testing web applications
Software testing is a process that participates in all life cycles of its development, which is engaged in the verification and planning of product requirements, preparation and implementation of changes in the developed system. As a result, there are inconsistencies with requirements and errors, the tools that were used are evaluated.
And if you distract and consider systematically, then the role of the tester actually lies in every person. If you look at the project as a person, then the roles
Quality Assurance is often considered synonymous with testing, but in fact it is a method of preventing errors, not finding them and fixing them. Quality in this sense is a characteristic of the product made.
Formally speaking, the tester must check even the technical task, project plan and analysis for consistency and clarity, and not just the code and interface of the program. Testing as an independent discipline in the general stream of software development appeared in the early nineties, when the audience (clients), quality and development time became more important.
The importance of software testing is especially important in critical systems. For example, the Therac radiation therapy apparatus(1985) irradiated with a lethal dose of radiation 6 people due to the lack of verification of the state of the unit in the code and validation. Or the rover in 1999 caused a loss of $ 125 million due to the lack of thought in a single measurement system between development teams. And even in Estonia there were problems with electronic elections due to poor load testing .
The basic principles of testing are embodied in the small ISTQB document and in the book Foundations of software testing.
The whole “charm” of the tester’s specialty is that, as a rule, this does not require great knowledge in programming, and people who are far from web-building can try themselves in this role.
Testing as part of the company's work on the project can take place either in the waterfall approach, where it is pulled under the tester only at the end of the project, or according to the agile methodology, where a finished version is rolled out with each iteration.
The testing iteration consists of the following items
Tests have their own levels, hierarchy and classes (hello RPG players!). You can check both different parts of the application (that is, by area, by page), and different properties of this section (that is, in depth, degree of quality). Here by the size of the sites can be divided into tests categories ...
Now, in fact, essentially testing ..
Testers as well as “technical task writers” for self-monitoring should always remember the abbreviation SMART, so that more practical people understand them correctly.
Hoc-of Ad (monkey) - random testing without a specific strategy
Exploratory testing (exploratory testing) - Product documentation if the study is largely absent
Grouping error equivalent grades ( EQUIVALENCE partitioning ). The most popular technique. Each input is checked for requirements (by the length of characters, possible values). In tests to minimize their number, classes of different inputs are combined to make the maximum coverage of errors.
Another technique exists primarily for integer values. Errors usually occur in boundary values , so the technique is called ( boundary value analysis) Two or three values are selected near the border. For example, if the maximum length of the header is 250 characters, then how will an error be generated with 249, 250, and 251 values?
More approximate to dynamical systems have methods of checking the state transition ( state transition testing ). The state depends on the input and on the previous state. States and transitions are described by graphs ( finite state machine ).
The decision table testing comes to mind by itself when the business rules in the application are very complex and require large mathematical calculations. Use tables to minimize the number of unique combinations.
Testing scenarios(use-case based) runs according to the scenarios created in the analysis, taking into account different roles, alternative flows.
Intuitively experienced testing (experience-based) as it is already clear comes with a person over time and allows you to find systematic errors in design, programmers, data flow, taking into account code changes.
The goal of structural techniques is to cover code coverage with tests as much as possible , and since the white box is used as an addition to scenario techniques, it is logical that there is maximum coverage of scenarios or, more precisely, their branching (statement & decision coverage).
Mathematically percent coverage = 100% * number of units covered / number of all units. This does not mean that everything will be tested, only the selected testing units ( coverage item ). The testing units for each role look different - for developers, these are database tables, functions in the code, a request to an external component, while for a tester and analyst, these are actions in a script, equivalence classes.
Rising above the testing units you can seeproposals and, accordingly, cover them ( statement coverage ). Good testing with black box techniques can cover up to 60-75% of offers. In terms of the code string is a specific
coating branches ( decision coverage ) have a higher level of hierarchy includes all types (if-else, switch, method calls and TP.). Black box methods can cover 40-60%. Cyclic complexity - a test algorithm known to all computer scientists for quadratic, exponential and logarithmic complexity.
The next degree of coverage includes possible paths.on which you can go through the branches. Theoretically, these paths are infinitely many if you take into account all the cycles.
The most familiar tools by the programmer are, of course, bug / issue tracking systems - Mantis , Bugzilla and Trac . Some IDEs can integrate with them so that commit actions in the version system repository (SVN / CVS / Git / Mercurial) can update the bug without working in the browser.
There are more general, industrial products, including requirements preparation, scenario grouping - HP (Mercury) Quality Center and IBM Rational Quality Manager
Automated interface testing is now most popular with Selenium , SilkTest , HP QuickTest Pro , QF-Test, Rational Robot, TestComplete , Watir , TestPartner. There are more exotic tests like soapUI .
Load testing is a little more complicated, because additional machines are needed (say, one machine simulates 300 users-connections) - it is usually impossible to run 3000 connections on one machine. Instruments:
Original article
And if you distract and consider systematically, then the role of the tester actually lies in every person. If you look at the project as a person, then the roles
- project manager is the spine and autonomic nervous system
- programmers are skeleton and motility
- analysts are sense organs
- testers are conscience
Quality Assurance is often considered synonymous with testing, but in fact it is a method of preventing errors, not finding them and fixing them. Quality in this sense is a characteristic of the product made.
Formally speaking, the tester must check even the technical task, project plan and analysis for consistency and clarity, and not just the code and interface of the program. Testing as an independent discipline in the general stream of software development appeared in the early nineties, when the audience (clients), quality and development time became more important.
The importance of software testing is especially important in critical systems. For example, the Therac radiation therapy apparatus(1985) irradiated with a lethal dose of radiation 6 people due to the lack of verification of the state of the unit in the code and validation. Or the rover in 1999 caused a loss of $ 125 million due to the lack of thought in a single measurement system between development teams. And even in Estonia there were problems with electronic elections due to poor load testing .
The basic principles of testing are embodied in the small ISTQB document and in the book Foundations of software testing.
The whole “charm” of the tester’s specialty is that, as a rule, this does not require great knowledge in programming, and people who are far from web-building can try themselves in this role.
Organizational process
Testing as part of the company's work on the project can take place either in the waterfall approach, where it is pulled under the tester only at the end of the project, or according to the agile methodology, where a finished version is rolled out with each iteration.
The testing iteration consists of the following items
- Planning. The goals, tools, time, order are determined. The most psychologically difficult part is to understand and remember how the system should work, especially when there is no technical task, documentation and a responsible project manager (see exploratory testing below)
Therefore, the tester has at least an internal, but the analysis consists of ...- Conditions
- Interaction Stories (cases)
- The embodiment in the form of a document (basis) with links to the specification, architecture and other documents (if they exist)
- Checks. In the case of inaccessibility and incomprehensibility - when and with whom to contact that the task would not be forgotten.
- Prioritization of test cases (usually 1 case = 1 requirement = 1 web page). What are the most critical scenarios? Which are the most risky in development?
- Grouping tests into kits (suite) by common modules
- Verification and comparison with requirements (actually search for errors)
- Writing a report (both bugs and the overall picture - coverage percentage)
- Rechecking a changed part (confirmation, retesting) or impact on unchanged parts ( regression testing )
- Completion. Archiving tests, saving lessons learned.
Test levels
Tests have their own levels, hierarchy and classes (hello RPG players!). You can check both different parts of the application (that is, by area, by page), and different properties of this section (that is, in depth, degree of quality). Here by the size of the sites can be divided into tests categories ...
- Modular ( unit-testing ). Semi-automatic testing, in the sense that the programmer has to write tests in very close contact with the class code that the unit test should check. It is not necessary to follow TDD, you can write at least for the most critical parts (financial transactions, etc.)
- Integration ( integration testing ) check the connections between different modules. It can be done both before the general integration of the system (module by module), and after (write, write, add - we will test it before the final acceptance test). Here again, close work with code and programmers.
- System - go through the entire system and find a discrepancy with general standards and specifications in general. As a rule, in the end, when the system is basically ready.
- Acceptance test usually takes place with the client during the transfer of the project and shows the willingness to work with clients. It is at this stage that the alpha versions appear (everything still lies probably with the developer on the machine, third-party testers are involved) and the beta versions (everything already lies with the client, potential clients are involved)
Test types
Now, in fact, essentially testing ..
- Functional answer the question " what " should the application do
- Non-functional requirements / tests answer the question “ how ”. Non-functional include the properties of interface convenience, reliability, data security, scalability, cross-platform and even economic feasibility (see ISO / IEC 9126)
- External (black box) / Internal (structural, white box). Structural testing checks the internal structure of the application as it is implemented, but needs an understanding of the code. The “black box” approach is based on the fact that the software has not yet been implemented or we do not know its exact operation, but there is a specification of how everything should work. Structural ( white box ) approaches test the application from the bottom up, concentrating on the maximum coverage of the code (rather than the input / output combinations in the interface). Typically, white box approaches are used in addition to the black box (how it should work).
- Regression testing - but has something broken due to this change? As a rule, this is what should be associated with unit testing for greater automation.
- Test support ( maintenance testing ) - the system is already working and we just update it (deploy & migration). Lately, migrations are also usually automated.
- Dynamic / static testing. Dynamic means interactive, work with a ready-made system or code. This is not always necessary and possible. Theoretically, you can test the documentation or check the consistency of the analysis, find errors from the logs, etc.
Testers as well as “technical task writers” for self-monitoring should always remember the abbreviation SMART, so that more practical people understand them correctly.
- Specific - an exact description of the requirement, without copy-paste of correspondence with the client, chat, data dump and monosyllabic sentences
- Measureable - the measured value of numbers, time, probability, users, location.
- Attainable - technically achievable requirements. Not everything is still possible technically, you can’t keep ahead of progress
- Realizeable - human-achievable human requirements for time, quality, knowledge, price
- Traceable - correspondence to the initial idea and purpose, logical consistency of several requirements.
Black box technician
Hoc-of Ad (monkey) - random testing without a specific strategy
Exploratory testing (exploratory testing) - Product documentation if the study is largely absent
Grouping error equivalent grades ( EQUIVALENCE partitioning ). The most popular technique. Each input is checked for requirements (by the length of characters, possible values). In tests to minimize their number, classes of different inputs are combined to make the maximum coverage of errors.
Another technique exists primarily for integer values. Errors usually occur in boundary values , so the technique is called ( boundary value analysis) Two or three values are selected near the border. For example, if the maximum length of the header is 250 characters, then how will an error be generated with 249, 250, and 251 values?
More approximate to dynamical systems have methods of checking the state transition ( state transition testing ). The state depends on the input and on the previous state. States and transitions are described by graphs ( finite state machine ).
The decision table testing comes to mind by itself when the business rules in the application are very complex and require large mathematical calculations. Use tables to minimize the number of unique combinations.
Testing scenarios(use-case based) runs according to the scenarios created in the analysis, taking into account different roles, alternative flows.
Intuitively experienced testing (experience-based) as it is already clear comes with a person over time and allows you to find systematic errors in design, programmers, data flow, taking into account code changes.
White box (structural) technique
The goal of structural techniques is to cover code coverage with tests as much as possible , and since the white box is used as an addition to scenario techniques, it is logical that there is maximum coverage of scenarios or, more precisely, their branching (statement & decision coverage).
Mathematically percent coverage = 100% * number of units covered / number of all units. This does not mean that everything will be tested, only the selected testing units ( coverage item ). The testing units for each role look different - for developers, these are database tables, functions in the code, a request to an external component, while for a tester and analyst, these are actions in a script, equivalence classes.
Rising above the testing units you can seeproposals and, accordingly, cover them ( statement coverage ). Good testing with black box techniques can cover up to 60-75% of offers. In terms of the code string is a specific
coating branches ( decision coverage ) have a higher level of hierarchy includes all types (if-else, switch, method calls and TP.). Black box methods can cover 40-60%. Cyclic complexity - a test algorithm known to all computer scientists for quadratic, exponential and logarithmic complexity.
The next degree of coverage includes possible paths.on which you can go through the branches. Theoretically, these paths are infinitely many if you take into account all the cycles.
Instruments
The most familiar tools by the programmer are, of course, bug / issue tracking systems - Mantis , Bugzilla and Trac . Some IDEs can integrate with them so that commit actions in the version system repository (SVN / CVS / Git / Mercurial) can update the bug without working in the browser.
There are more general, industrial products, including requirements preparation, scenario grouping - HP (Mercury) Quality Center and IBM Rational Quality Manager
Automated interface testing is now most popular with Selenium , SilkTest , HP QuickTest Pro , QF-Test, Rational Robot, TestComplete , Watir , TestPartner. There are more exotic tests like soapUI .
Load testing is a little more complicated, because additional machines are needed (say, one machine simulates 300 users-connections) - it is usually impossible to run 3000 connections on one machine. Instruments:
- Minor - Apache ab, httpperf , openload , Neoload , hammerora
- Large - Apache JMeter , IBM Rational Performance Tester , HP LoadRunner (Mercury), Oracle Application Quality Management (Empirix e-Load)
Original article