Requirement Traceability Matrix In Testing

Whenever we purchase any product such as any furniture, electronics items, or even food online, it provides us with a tracking option of the order so that we can track what we purchased to know the state of our order currently, such as where is the order, when it is going to deliver etc. Imagine what will happen if there is no traceability of what you have ordered! We would not know if the order you have placed and the product going to deliver is the same or not when it will get delivered etc. many questions may arise on top of our mind. Again back tracking, such as return or replacement of products would not be as simple as with correct traceability or tracking. Real-time tracking gives us peace of mind, reassuring us that our “investment” is on the way and not lost somewhere. So the question is, as an individual, if we are getting affected by our small online order, imagine the level of tracking a software product would require!

Traceability makes life easier in all aspects. Trust is good, but traceability is better as it keeps the transparency between different members participating in software product delivery and its customers. On a similar lane let’s now discuss how requirement traceability matrix (RTM) plays an important role in software testing. We will discuss the following topics in detail which will give you a broad understanding of what requirement traceability matrix is. By the end of the article you will be able to prepare your own RTM document with ease.

What is Traceability Matrix (TM)?

Traceability Matrix is the driving factor in ensuring sustainability of a software product.

Coming to the definition, according to wiki “in software development, a traceability matrix (TM) is a document, usually in the form of a table, used to assist in determining the completeness of a relationship by correlating any two baselined documents using a many-to-many relationship comparison”.

As per ISTQB, Traceability is the ability to identify related items in documentation and software such as requirements with associated tests. So, the “test conditions” should be able to be linked back to their sources in the “test basis” which is called Traceability Matrix.

i. Test Basis: The process of looking at something that can be used to derive the information is called test analysis whereas the basis we look to derive the information is called “test basis”. It could be a system requirement, a technical specification, the code itself, or a business process.

ii. Test Condition: We look at the test basis in order to determine what could be tested, these are the test conditions. A test condition is something that we could test.

iii. For example, if we are looking to measure coverage of code decisions, then the test basis would be the code itself and the list of test conditions would be the decision outcomes (True or False).

iv. Similarly, if we have requirement specifications, the table of contents can be our initial list of test basis. This brings the topic Requirement Traceability Matrix. Let’s discuss it.

What is Requirement Traceability Matrix (RTM)?

Requirement Traceability Matrix is a testing document or an artifact that keeps track of all the customer or user requirements and the details of the test cases mapped to each of those requirements. It serves as documented proof that all the requirements have been accounted for and validated to achieve their end purpose.

Why do we need Requirement Traceability Matrix?

As pointed out by Dr. William C. Hetzel,1988, “a good way to understand requirements better is to try to define tests to meet those requirements”. For example, if we are testing a customer management and marketing management system for a mobile phone company, we might have test conditions that are related to a marketing campaign, such as the age of the customers such as whether he is a pre-teen, teenager, young, adult, mature, what is customers gender, their address, postcode or zip code and purchasing preference such as contract, EMI, pay-as-you-go, etc.

i. Consider another example, the requirements for a given function or feature have changed, some of the fields now have different ranges that can be entered. Which tests were looking at those boundaries? Which regression suite you need to execute etc. They now need to be changed. How many tests will actually be affected by this change in the requirements? These questions can be answered easily if the requirements can easily be traced to the tests. You can refer the article How RTM Relates to Regression Testing? to know more on how requirement traceability matrix relates to regression testing (refer Getting Started With Regression Testing: Challenges, Strategies, and Best Practices to know more about regression testing.)

ii. A set of tests that have run OK in the past has started to have serious problems. What functionality do these tests actually exercise? Traceability between the tests and the requirement being tested enables the functions or features affected to be identified more easily.

iii. Before delivering a new release, we want to know whether or not we have tested all of the specified requirements in the requirements specification. We have the list of the tests that have passed - was every requirement tested?

Traceability Matrix between manual and automated test cases

Traceability between manual and automation test cases is again a vital activity that a software tester should do. As the application evolves, changes in manual tests come in and scripts need to be updated as per current requirements or new enhancements, bug fixes, etc. Having this document would make the way easier for the person updating scripts if any prior discrepancy was written with reasoning readily available. Secondly, during regression, it would be very clear which areas automation is not looking at and which the manual tests might want to look into.

Tools that can help:

Creating test cases, traceability matrix, and automation scripts should be tightly linked but how to achieve this? A way to solve this is Testsigma tool, which helps to maintain a traceability matrix between manual and automated test cases. It tracks the requirements with automated test cases and integrates with test case management tools. We will also discuss about various tools available in the following document.

Testsigma lets you automate your tests for web, mobile, APIs and desktop from the same place, thus, you don’t to have different tools for different platforms.

Advantage of Requirement Traceability Matrix

Depending on which industry or platforms you work in, requirements management and traceability can help teams to do many things, including:

i. Assess risks and the overall impact of a change before it's made.

ii. Manage changes throughout the process and avoid scope creep.

iii. Ensure quality standards are met to achieve compliance.

iv. Control development costs by avoiding costly engineering rework.

v. Guarantee adequate test coverage of requirements before a release.

vi. Improve visibility into the process for the entire team and stakeholders.

vii. Shows the missing requirements or conflicts in documents.

viii. Effort estimation

Disadvantages of Requirement Traceability Matrix

Despite its usefulness, in some cases, many organizations fail to implement requirement traceability matrix practices effectively. We are going to discuss few points:

i. Manual requirement traceability methods are labour-intensive and error-prone.

ii. Manual requirement traceability methods are limited as you integrate more tools

iii. A requirements traceability matrix doesn’t scale or age well

iv. It is time-consuming to make sense of the information

v. Manual requirements traceability methods do not fit an agile team

Types of Traceability Test Matrix

Traceability can be either horizontal through all the test documentation for a given test level (eg. System testing, from test conditions through test cases to test scripts) or vertical through the layers of development documentation(e.g. From requirement to components).

On a high level, Requirement Traceability Matrix, can be classified into three different types: forward traceability, backward traceability, and bidirectional traceability. Let’s discuss each one briefly.

1. Forward Traceability:

This is also called Horizontal Traceability matrix.
Forward traceability is used to map the requirements to the test cases.
This is done before the test case execution is started.
This establishes that every requirement is being tested from top to bottom and checks that the project trajectory’s is running smooth.

2. Backward Traceability:

This is also called as Vertical Traceability matrix.
Backward traceability is used to map the test cases with the requirements.
This is mapped after the test cases execution is done.
Backward traceability is used to check that we are not increasing the space of the product by enhancing the design elements, code, test other things which are not part of business requirement needs.
When the scope of development is not changing frequently, we use this.

3. Bi-directional Traceability:

Bi-directional Traceability is the combination of Forward traceability and Backward Traceability matrix. That means this is the combination both forward and backward document.
Bidirectional traceability Matrix is useful because it make sure each requirement has its corresponding test case.
This also evaluates the modification in the requirement which is occurring due to bug in the applications.

How to create Requirement Traceability Matrix

In the planning and preparation phases of the testing, testers should review and contribute to test plans as well as analysing, reviewing and accessing requirements and design specifications. Testers should understand the intended behaviour, the problem the system will solve, the process it will automate and so forth, in order to spot improper behaviour while testing and recognize the “must work” functions and features. As we saw different benefits of requirement traceability matrix, one of the very common question that comes next to our mind is what should go into the requirement traceability matrix?

Define a clear goal on why you are creating the current Requirement Traceability Matrix.
Collect all the available requirement documents/artifacts.
Figure out best traceability matrix tools if available. Else prepare a requirement traceability matrix template.
Allocate a unique Requirement ID against each and every requirement.
Create Test Cases for each and every requirement.
Link these test cases IDs to the respective Requirement ID.
Update the Requirement Traceability Matrix timely based on the current status or any changes done due to bug fixes etc.

Requirements Traceability Matrix (RTM) Template:

Requirements Traceability Matrix template itself serves as vehicle for communicating with other members of the project team, testers, peers, manager, customers and other stack holders .

Below we will see an example of Requirement Traceability Matrix template. Note depending on each product and organization this Requirement Traceability Matrix document may varies.

Tips On Creating A Requirement Traceability Matrix
ID	A unique ID number used to identify the traceability item in the requirements traceability matrix.
Associated ID(s)	This column should contain the ID of any associated utilities used for requirements tracking such as a repository, pipeline document, etc.
Technical Assumption(s) or Customer Need(s)	This column should be populated with a description of the technical assumption or customer need linked to the functional requirement.
Functional Requirement	This column should be populated with a description of the functional requirement.
Status	This column should be populated with the current status of the functional requirement.
Architectural/Design Document	This column should be populated with a description of the architectural/design document linked to the functional requirement.
Software Module(s)	This column should be populated with a description of the software module(s) linked to the functional requirement.
Test Case Number	This column should be populated with the test case number linked to the functional requirement.
Tested In	This column should be populated with the module that the functional requirement has been tested in.
Verification	This column should be populated with a description of the verification document linked to the functional requirement.
Additional Comments	This column should be populated with any additional comments

Sample Requirements Traceability Matrix template:

Requirements Traceability Matrix (RTM) Tools

Now that we understand the Requirements Traceability Matrix’s importance, how should you go about looking for tools to help you manage and trace your project’s requirements? There are a number of key features that you should keep an eye out for, which can drastically improve your work efficiency and the quality of your product.

Multiple views of requirements, including grids, boards and documents
Ability to trace requirements to tests, code, builds and development activities
Flexible and customizable reporting, including requirements traceability reports
Requirement workflow and signoff with support for electronic signatures
Ability to version requirements, with support for baselines and snapshots
Audit trail of all changes made to requirements
Rich content editing, including formatted text and diagrams
Support for different types of requirements such as user stories, use cases and scenarios
Ability to attach files, images and other documents to requirements
Collaboration features including chat, email, comments and discussions

In excels it would be a nightmare to continuously work and update everything. This is an exhausting process and it is prone to human errors. A requirement traceability matrix tool would not only automatize all that manual work, but it would also open the door to further advantages, like compliance verification, impact analysis, or requirements validation. Here, I will give you a few examples of different Requirement Traceability Matric tools available.

1) Xebrio

Xebrio tracks individual requirements throughout the lifecycle of your project with multiple levels of stakeholder approvals & collaboration capabilities, the ability to associate requirements to tasks, milestones, and test cases, and a transparent yet detailed process for requirement change management, thereby guaranteeing requirement traceability.

2) Jama Software

Jama Software provides the leading platform for requirements, risk, and test management. With Jama Connect and industry-focused services, teams building complex products, systems, and software improve cycle times, increase quality, reduce rework, and minimize effort proving compliance.

3) Atlassian Jira

JIRA Core is a requirement management and business analysis tool. It helps every business person to plan, track, and create a report of their work.

4) Visure Requirements:

Visure Requirements tool is provided by Visure Solutions, which is focused on business-critical and safety-critical industries. Its Visure Requirements tool provides complete traceability.

Conclusion

“A traceability solution is only as good as the data it provides,” DiPalo said. “The data itself is an integral part of the product quality, so it needs to provide an accurate picture of the state of the developed product and whether it meets the customer requirement or not. In this article, we have discussed Requirement Traceability Matrix elaborately including why it is required, different types, its advantages, and disadvantages, selecting the correct RTM tools, etc. Remember a good Requirement Traceability matrix is something that can track and trace every requirement need and aligned test case mapping at any stage of the software development life cycle.

Requirement Traceability Matrix In Testing - An Overview