1. Requirement

3.4.8 The project shall provide and maintain bidirectional traceability from the Software Test Procedures to the software requirements.

1.1 Notes

NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.

1.2 Applicability Across Classes

Class G is labeled with "P (Center)." This means that an approved Center-defined process that meets a non-empty subset of the full requirement can be used to achieve this requirement.

2. Rationale

Software test procedures are created to verify the software requirements for a project. In order to ensure that all requirements are verified via the test procedure set, the requirements need to be linked to the test procedures which verify them.

Traceability matrices help ensure that test procedures verify one or more software requirements by mapping those procedures back to one or more software requirements. Traceability is also used to assure that the necessary level of test coverage is achieved, i.e., that there are sufficient tests to verify the requirements have been correctly implemented in the software.

Traceability links between individual requirements and other system elements, including, but not limited to, test procedures, are helpful tools when evaluating the impact of changing or deleting a requirement. When a requirement is changed, traceability can help identify the affected products, including design, documentation, source code, tests, etc. (NASA-GB-8719.13, NASA Software Safety Guidebook). ²⁷⁶

Traceability is important because it can point out software requirements that are not tested (i.e., missing tests) as well as tests that do not serve to test requirements (i.e., extra tests). 

3. Guidance

Before starting the traceability activity, it is assumed that the documents being traced (e.g., requirements, design, code, test data, etc.) have been approved.

Using a matrix such as the one shown below ³⁵⁶  allows a single exercise to show traceability both forwards and backwards. The matrix is completed left to right early in the appropriate phase in the project life cycle. As each column is completed, the forward trace is extended to the next set of products. Simply starting with a column, such as the LLD (low-level design) section and looking at the data in the columns to the left, shows the backward traceability from a LLD element to its parent HLD (high level design) element and back to the parent requirements.

While traceability matrices are not the only method for capturing bidirectional traceability, they are the most common. Traceability matrices can be included in the documents to which they apply, such as a test plan, or they can be combined into a single matrix covering higher level requirements, software requirements, design, code, and verification. General guidance for creating a bidirectional traceability matrix includes the following suggested actions:

• Create the matrix at the beginning of the project.
• Uniquely identify the elements in the matrix (requirements identifiers, design document identifiers and paragraph numbers, function identifiers, test identifiers, etc.).
• Keep the matrix maintained throughout the life of the project.
• Assign responsibility for creating and maintaining the matrix to a project team member, since managing the links/references can be a labor-intensive process that needs to be tracked/monitored.
• Maintain the matrix as an electronic document to make maintenance and reporting easier.
• Create the matrix such that it may be easily sorted to achieve/convey bi-directional traceability.
• Ensure a review of the matrix at major phases/key reviews of the project.

A bidirectional traceability matrix can be manually created and maintained, or may be a by-product of a requirements management tool. The tracing system needs to be chosen based on project complexity and the number of requirements. Check with project management to see if a requirements management tool exists for the local project that is capable of producing a bidirectional traceability matrix.

Keep in mind that a single requirement could trace to multiple test procedures. The reverse is also true, test procedures could trace back to multiple requirements, so the relationships identified in the matrix are not required to be one-to-one. The matrix needs to contain no missing relationships, i.e., empty cells in the matrix, as those indicate a problem with the set of test procedures which need to be designed such that every requirement is verified.

It is possible that new requirements may be generated during design and implementation. When that happens and the requirements are confirmed as being within the scope of the project (not expanding the scope or "gold plating" the system by including unnecessary functionality), the traceability matrix needs to be revised to include the new requirements and the mapped design elements, implementation (source code), and test procedures.

To ensure full traceability between requirements and tests, it is important to trace test cases, test scripts, test data, and other supporting test information not already found in the test procedures to the relevant test procedures. This level of trace information may or may not appear in a traceability matrix. The test procedures, test cases, test scripts, and test data can include the proper links and references to ensure full traceability among all the elements of the tests.

Key aspects of tracing test procedures include:

• Ensure that tests for safety-critical functions are clearly identified either through the traceability matrix or through test procedure documentation.
• Trace each requirement and functional specification to one or more test cases (Manager's Handbook for Software Development ⁰³¹).
• If not already done, trace unit tests to source code and to design specifications.
• Trace integration tests to high-level design specifications.
• Trace system tests to software requirement specifications (SRS). ¹²⁷

Additional guidance related to bidirectional traceability may be found in the following related requirements in this handbook:

4. Small Projects

For small projects without access to a requirements tool that includes tracing features and with time/budget limitations preventing them from acquiring a new tool and associated training, requirements tracing may be done with a spreadsheet (such as Excel), a simple database (such as Access) or a textual document. It is very important that the project be diligent about keeping such traces up to date as these methods do not include automatic updates when requirements, design elements, or other relevant documents change. (NASA-STD-8719.13B, NASA Software Safety Standard). ²⁷¹

Value-based requirement tracing may be an option for projects with small budgets where traceability of safety-critical requirements is the priority. "Value-based requirement tracing prioritizes all of the requirements in the system, with the amount of time and effort expended tracing each requirement depending on the priority of that requirement. This can save a significant amount of effort by focusing traceability activities on the most important requirements. However, value-based tracing requires a clear understanding of the importance of each requirement in the system; it may not be an option if full tracing is a requirement of the customer or the development process standards used for the project." (Why Software Requirements Traceability Remains a Challenge ²³⁷)

5. Resources

5.1 Tools

Tools to aid in compliance with this SWE, if any, may be found in the Tools Library in the NASA Engineering Network (NEN).

NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN.

The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool. The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.

6. Lessons Learned

A documented lesson from the NASA Lessons Learned database notes the following:

Software Requirements Management. Lesson Number 3377: "The ability to manage and trace software requirements is critical to achieve success in any software project and to produce software products in a cost-effective and timely fashion. ⁵⁷⁶

• "Manual methods for management of software requirements are ineffective and inefficient, contributing to excessive costs as well as schedule delays. Aspects of the management of software requirements include the elicitation/specification, analysis, development, tracking, and changing of software requirements used during the implementation and sustaining phases of the software life cycle. Management and traceability of software requirements are critical to the success of producing reliable, high-quality, and safe software products that meet end-users requirements and needs in a cost-effective and timely fashion.
• "Cost and schedule impacts that result from incomplete, incorrect, or changing software requirements increase the later they occur in the software life cycle.
• "Current software technology, processes, and tools provide innovative automated methods to facilitate optimum management of software requirements (e.g., IBM Rational DOORS, IBM Rational RequisitePro, Cradle requirements management software).
• "Additionally, a collaborative relationship between the customer using the software and the developer providing the software is paramount to the success of the software project. More specifically, the users/customers must effectively define and accurately communicate their requirements to the developer. For example, the user's defined requirements should be clearly stated and unambiguous, concise, complete, autonomous, able to be implemented, and testable."