Test Documentation in QA: Principles and the Best Practices

Testing is an essential aspect of any product. Without testing, you cannot be sure of what you are giving the customer or the user. At any time, an unexpected problem may arise that is not obvious when looking at the code. But go a little bit beyond the intended use cases, and you can immediately see if there are problems with high load, big data, cyber-attacks, etc. And, of course, you don't want to discover these problems after the software has been launched, especially in feedback from real users. It is necessary to take testing and maintaining test documentation seriously to prevent this.

A bit about testing

So making a tester out of a customer or user is a terrible idea for various reasons. Some of them are obvious, and you don't have to be a prominent expert to see their seriousness. Some, on the contrary, may seem insignificant, but only in the process do you realize how important they are.

First of all, the ethical reason is that it's not fair. People are giving their real money, which they have earned by spending their real-time, and they aren't getting what they're paying for.
Further, the commercial reason is that the customer can lose profits if the product causes problems rather than provides a solution. And the regular user will quickly lose interest in your product and find an alternative that works better.
The reputational reason is that you will not be able to build a trusting and loyal relationship with the customers in such a situation. Most likely, they will not devote their lives to destroying your company. But they can break relations with your company and dissuade their closest friends from cooperating with you, and they will dissuade their friends and so on. In the case of regular users, everything is the same. Only the effect will be even faster and broader.
The legal reason is that the customers can even take the company to court for violating the contract terms because of huge losses or a remarkably unfinished product. They can demand a refund, compensation for losses, and much more in such a case. It can make your company unprofitable for a long time or even bankrupt.

What is test documentation?

Test documentation is a type of documentation that describes the process, objectives, and results of software testing. It can also include information on the environment, setup, and configuration required to perform testing. Test documentation is used to communicate the details of a test plan or strategy to stakeholders, developers, and testers.

Of course, some companies argue that test documentation is an optional process that only adds cost to development and works more for the image of the software company than for providing real value. But the truth is that test documentation is an integral part of the test process that should precede, accompany, and finalize any test.

At Mad Devs, for example, we believe quality test documentation is obligatory, both during the full-scale product testing phase itself and during all other stages where intermediate tests are conducted. Thanks to many of our articles, you can learn more about the testing process. Our best developers share their expertise, and our writers help deliver this most pleasantly and understandably.

What are the main goals of test documentation?

The main goals of test documentation are:

Guide future testing efforts and allow stakeholders to understand testing outcomes and make informed decisions about the software.
Help to prevent gaps or overlaps in testing coverage and ensure that all necessary tests are performed.

Well, we've looked at why tests and test documentation are so necessary. But what are its more specific goals?

Make testing transparent - test documentation shows all aspects of testing, from goal setting to test methods and results. It gives the customer a clear picture of what work was done and its results. It's also beneficial for other testers to whom the task is handed over and the developers who have to solve the problems they discover.
Improves team communication - communication errors are significantly reduced when a team has a single source of information on any process. Even if everyone understands the situation differently, there is always something to reference and discuss. There is also no need to make redundant calls to get the necessary information because it is constantly updated, and everyone knows where to look for its current version. And, of course, it helps a lot when new people come into the team or when the testing team changes in general, which without quality documentation, can be impossible.
Analyze and improve the testing - only when you have fixed approaches to testing can you analyze them for effectiveness or compliance with software testing standards and best practices. Then you can document whether the change in approach has resulted in a measurable increase in efficiency or vice versa. So only with test documentation can you make an impact on your testing processes.
Meet deadlines and budgets - when testing is properly documented, the risk of unexpected problems is dramatically reduced. It means that the cost of solving them also becomes much lower. For example, under heavy loads, your server does not correctly collect user metrics, the service hangs, and hackers gain access to users' data. Without testing documentation, testers and developers will have to spend extra time finding out the cause of the problem. Marketers will have to do extra work to recover reputational losses. You will have to spend a separate budget for all their efforts. A well-written and documented test helps you discover, fix, and describe it upfront, and it's cheaper for you in every sense.
Capture your own experience - well-done test documentation is a capture of real development experience based on which you can do a lot of valuable things. You can use it to train new team members, find and share best practices, raise reasoned discussions on media platforms, and so much more. Of course, this requires extra separate work because you can't just share internal documents and codebase.
Prove the seriousness of the development approach - good test documentation is an excellent demonstration of how seriously you take your services and applications' security, stability, and speed. How fundamental testing is done, whether it takes the best approaches, and how different cases are tested. In this way, you show how much attention you pay to many aspects that may not seem significant but may be critical.

What are the different types of test documents?

There are a lot of test documents, and they can meet a wide variety of standards. The most common type of test documentation is the test plan. A test plan outlines the approach that will be taken to test a particular software system. It includes information on what will be tested, how it will be tested, and who will be responsible for conducting the tests.

Other types of test documentation include test cases, test scripts, and defect reports. Test cases describe the specific steps that need to be taken in order to test a particular feature or functionality. Test scripts are more detailed instructions that outline exactly how a test should be conducted. Defect reports document any errors or bugs that are found during testing.

Types of test documentation depend on the particular company, product, and customer. Let's take a closer look at the most common types of test documentation.

Internal test documents

Internal test documents are required for work tasks and for use by the development and testing team members. It contains the goals, methods, and results of testing at a technical level and sometimes requires a technical background to understand them fully. But because they are designed to make life easier for everyone, they can generally be understood by anyone from both sides. By project managers, developers, and testers of the development company and business analytics, project managers, and technical directors of the customer company.

Test policy - it is a document that outlines the company's rules which must be adhered to when performing testing. For example, whether you can use your own devices or only corporate devices during testing, data from public sources, etc.
Test strategy - it is a document that outlines the main aspects of testing, in particular, at what levels of the project it will be performed. During testing, developers, designers, and managers can return to this document at any time to ensure that testing is still within the original strategy and allocated area.
Test plan - it is a document that details all major aspects of the testing project, such as scope, approach, resources, schedule, testing members, and their activities. Since it is the most basic, this document is used most often, distributed to the entire team, and shared with customers.
The test scenario - it is a document that defines the testing process from its beginning to its end for a specific part of the product. It describes a number of test cases dependent on each other for testing.
Test case - it is a document that describes exactly how a particular part of the product is tested, such as its logic, function, interface element, etc.
Test data - it is a document that describes the data needed for testing to bring the application's performance closer to real-world use. For example, it can be sets of generated users, documents, media files, metadata, and anything else that the application has to work within the real world.
Test log - it is a document with different test cases for a particular type of test to be performed during the test. It also records the test results, which provides detailed evidence for a summary report and allows you to reverse engineer the test if necessary.
Traceability matrix - it is a table that contains various test cases with their requirements and execution status. It is used to track requirements and their compliance throughout the software development lifecycle. It can be used to track the process from design to coding and vice versa regarding each type of testing.

External test documents

They are needed to visually present the final results because using various types of visualizations makes them easier to understand. They are often provided to customers and may be provided to users in some form.

Bug reports - it is a document that describes a particular bug in an application in all its possible aspects. It is a very important and frequent report that directly helps improve the product in specific aspects.
Test summary reports - it is a document that summarizes the results of a test cycle. It may contain the cost of defect detection, test suite efficiency, testing efficiency, a ratio of rework and verification efforts, etc.
User acceptance report - it is a document that contains the results of testing before submission to the vendor for compliance. This ensures that the developers' and customers' visions remain identical during development and testing and that the technical implementation fully meets them.

How to write a test document?

It is worth noting that each company's approach to maintaining and selecting the types of reports may differ. Since many of them may be interchangeable in some aspects, companies may use these reports slightly differently.

Of course, only basic test reports are listed here and shown in the general order. Depending on its product, each company can combine reports, change their order, and add or exclude some of them as needed.

First, a test strategy document defines the overall approach to testing the application. It specifies which parts of the application will or will not be tested and according to what requirements it will be done.
Then a test plan is made in which all aspects of testing are to be specified in detail, starting from resources and tools assumed for this purpose and finishing with a list of application functions to be tested and their testing schedule.
Then a test scenario document is made where you specify in detail what exactly will be tested and how many test cases are required to test a particular element of the application from start to finish. There can be one scenario for each application element or several scenarios for one element.
In parallel, a data test document is made, which specifies all necessary data to test each test case.
Then a document of test cases is made, each of which describes in detail how each particular element of the application will be tested.
After that, the Traceability Matrix document is made, directly linked to the rest of the testing documents, and helps get access to the updates.
And only then are external reports done. After that, they should be updated regularly and run in parallel. Only then will they show their results in the best possible way.

For example, we at Mad Devs use almost every one of them, we update them regularly, and it does make a huge benefit for our company. Every one of our customers knows very well that we have a quality report for every process we do and for every type of testing in particular. So our customers always know what we do and how we do it, and they see the actual result, for which they return to us again and again. You can see this by looking at their reviews.

Manual Testing vs. Testing Automation: Which is better to Use?

The choice between manual or automated testing is the basis for a process that contributes to the project's overall success. Check the article to know which is better.

What are the best practices for software testing documents?

Keep your documentation up to date - a common mistake is to stop updating your documentation, even if you do all the documentation very well at the start. And this seems insignificant, but only until you need it.
Keep your documentation in one place - documentation should not be scattered in different places in an attempt to keep it close to the appropriate codebase. This may seem like a good idea, but it can confuse team members and create a mess in a practical sense.
Keep documentation in a safe place - don't keep any documentation in the public domain that could negatively impact business goals. In that case, you need an encrypted repository and the ability to provide access to certain individuals.
Use standard templates for documentation - yes, you may have to use custom solutions sometimes, but it's best to use universal documentation solutions such as Word and Excel. This will make it easy for anyone interested to open and read them without installing additional software or other complexities.
Make your documentation detailed - there is nothing worse than poor documentation. Seriously, even its absence does not say anything about the company, but poor documentation immediately negatively affects it. Make it well-structured, meaningful, detailed, and clear.

Summary

We can conclude that testing is an essential part of software development. And an integral part of good testing is keeping quality test documentation. It provides a roadmap for testing and helps ensure that quality assurance goals are met.

The set of test documentation may vary from project to project. But there are unchangeable test documents, the quality maintenance of which gives the company, its employees, and customers many advantages.

Bekmyrza Dzhekishev

ex Quality Assurance Engineer