Create clear test reports

Introduction

This article will be useful for specialists not only in testing, but also from other areas.
I think everyone understands that reporting is often the part that is required on a project, but it is always problematic to write it up. Everyone, sooner or later, is faced with the problem “how to describe it?”, “What to write?” And most importantly “why and who will read it?”.
In fact, a report is an important and concise form of transferring information from an executor to a customer. This is a response to its technical requirements and at the same time information about the work done.
Today we’ll talk about testing reports. In the article you will find emphasis on important points when creating reports.

Clear test report

Creating a clear test report (test-report) in practice.

To begin with, let's recall the definition: A
report is a document containing information about the actions taken, the results of the work done. Usually it includes tables, graphs, lists, simply describing the information in the form of text. Their proportion and content determine the usefulness and clarity of the report.
It is important for us to understand for whom, for what and in what conditions we are doing this and how much it will improve the perception of the information we present. It must be remembered that each action has a specific goal. In the case of the report, it is important for us to understand for whom, for what and in what conditions we are doing this.
Let's look at the diagram:



Analytical sections - this is our report. In it we give an analysis of our work and evaluation of the tested product.
The type of company, in an ideal situation, should not affect the quality and semantic capacity of reporting. In the real world, unfortunately, the reporting of outsourcing companies is, as a rule, better and more capacious than the reporting of full-time testing departments (there are pleasant exceptions).
We, like any other outsourcing company, are forced to pay great attention to the quality and transparency of reporting, because it is the key metric for evaluating our work visible to the customer.
The reporting itself can be divided into final and regular - daily, weekly, monthly, versioned (for each version of the product), etc. The differences lie in the depth of the time sample.
So, before writing a report, first we need to decide who we are writing it for.

For whom are we generating a report?

When creating a report, it is important to understand who it is created for and who will read it.
Based on the priorities of the target audience, we must determine what information the report should contain. Accordingly, during the project, information should be consolidated in the direction that needs to be reflected.
We can distinguish three groups of target audiences:
1. Technical users - Test-manager. Their priority is to understand the testing process, what problems arise, how they are solved, the construction of the testing process itself, a description of the methods and technologies used.
2. Product ManagerThey are Product Managers. Their focus is on deadlines, squeezing out of test results without unnecessary technical details, and on general statistics (digital and comparative metrics).
3. Business users . Usually these are the people who make decisions to complete the testing. They determine the quality of the work done. For them, first of all, the final result is important in the most concise and clear format (yes / no), visual presentation of information (graphs, charts), expert opinion on the possibility of releasing the product in an industrial environment, etc., without going into details .

Output:Writing a report that will suit all groups is almost impossible. Before writing a report, be sure to determine the target audience. Depending on it, the content will be very different in its structure and contain different details necessary for a particular group.

What is the depth of the time sample?

Reports can be divided into two types with respect to time:

1. (Weekly, daily, monthly) / interim.
In general, this is almost the same final report, but with changed focus priorities and a reduced depth of time sampling. It must contain two main metrics:
- Assessment of the degree of product readiness.
- Evaluation of the work carried out on testing for the time between reports (progress).

This report should show what the dynamics of your work are.

Important to remember that progress- the value is not constant, but dynamic, it is determined by comparing the status of the project last week and the present. Accordingly, progress is this set of metrics that make it possible to understand the state of the project.
They are created for each project individually, based on the goals that are set for successful testing. Metrics are set when creating a TC (test cases), passing a TC (failed / passed), detecting defects (criticality). They allow you to easily and quickly enough make up the overall comparative picture of the project. If you, for example, use TestLink, you understand that metrics allow you to quickly select problems, compile statistics of failed TCs, etc.
This information is useful and necessary for Product Manager, it is composed and controlled by Test-manager, as well as QE and SQE.
There is another important and frequently used type of temporary report - versioned (iteration report).
It is similar to the final one. It describes the tasks that were performed by the testing team for a specific version of the product.

2. Final / final.
In the final report, it is important to show a general view of the work done (in the context of established metrics) and the evolution of the product.
Also, it is necessary to give comprehensive information about the status of the product at the moment (the number of remaining uncorrected errors, whether the product is fully tested or requires an additional testing cycle, an assessment of the possibility of releasing the product to the "outside world", etc.)

Output:Keep statistics using metrics throughout the project. She will help you to provide any information to the customer at the right time and relieve you from fear of the questions “What exactly did you do in the fourth week?” And “What do we have with the deadlines?”

What methods of presenting information and data to use in the report?

When a technical specialist writes for another technical specialist, the question of the application of certain methods of reflection of information arises rarely. Terms, formulas, professional slang - this is familiar and understandable. It is much more difficult to write reports for people who are relatively far from the specifics of testing.
For business users , often use the presentation of information in the form of graphs. They clearly show how much the product is ready for release into the industrial environment, by how many percent the project is completed.
This can be, for example, a graph of the passed TCs on modules. It will clearly show how much work has already been done in each module and will help isolate problems.



Also, a graph of the relationship of created tickets (detected bugs) and closed (fixed bugs) can be very useful. Not for nothing, it is the main one in many task trackers.
In the case of programmers' productive work on fixing defects and writing high-quality code, the critical error curve should tend to the bottom with the release of a new release, while the priority and importance of errors also decreases.
But, if developers or testers pay little attention to existing defects, then the curve of closed bugs grows more slowly than not closed ones.
In the ideal case, the curve of unclosed bugs (found but not fixed) should converge with the curve of fixed bugs. In other words, the final release requires that all defects be fixed. If this is not so, then the management may decide to extend the development and testing in order to eliminate all defects or release the product in the “prod”, taking on possible risks.
In addition to the schedule, it is necessary to draw up a pivot table. The graph is built on the basis of these data.
Here is an example of a table on the basis of which a graph of the completed TCs for the modules was built:



Conclusion: A schedule for business users is a mandatory part of reporting. It is informative, accessible and understandable to the end user, demonstrates the dynamics of activity on the project or, in the worst case, stagnation.
Also, the use of graphs in reports for any users and technical experts is advisable when you need to quickly and clearly compare numbers and show dynamics.

WHAT DO I NEED TO INDICATE IN THE REPORT ALWAYS?

Different types of reports may seem very different.

Nevertheless, they have similar features and data that should always be indicated.
Here they are:
1. Team composition;
2. Deadlines for which a report is compiled;
3. Description of testing processes;
4. Changes to the test model, addition to the TC;
5. The percentage of completed TC;
6. Critical and blocking problems and measures taken to eliminate them;
7. The results of the regression (plus an emphasis on the remaining problems);
8. Plan for the next iteration \ week \ month;

Points 3, 4, 6, and 8 should be written with an eye to the target audience of the report.
The seventh point should be indicated when the "regression testing" was carried out. Usually this item appears in "versioned" reports.

Paragraph 8 is excluded from the final report.

Conclusion

So, we understood our target audience, designated the period for which we will write the report, defined the content and blocks. In fact, this is practically all that is needed to form an understandable document that will surely find a response in the heads of those to whom it is addressed.
Write your reports in detail, competently and with pleasure, because a good report is at least a third of the work and the only part of it that is visible to someone other than testers and programmers.

Author: Efremova Daria