1. Requirements

5.4.6 The project manager shall collect, track, and report software requirements volatility metrics.

1.1 Notes

NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.

1.2 History

1.3 Applicability Across Classes

Key:    - Applicable | - Not Applicable

2. Rationale

Requirements volatility is one of the leading causes of the software development effort not completing on schedule and budget.  Software requirements volatility is one key factor in assessing the status of a software project.  The later in the project requirements changes occur, the more impact those changes can have on project completion on time and within budget. 

3. Guidance

3.1 Applicability

This requirement applies to all NASA centers for Class A and B software. Class C software development efforts may also consider collecting these metrics, especially if the development schedule is very tight. See also Topic 5.01 - CR-PR - Software Change Request - Problem Report, 

3.2 Impact Of Requirements Changes

Requirements changes can occur throughout the entire lifecycle. Requirements changes for clarification or to add something that was overlooked are usually caught fairly early on in the lifecycle. In general, by PDR most requirements have been identified and refined. The cost of these changes, before PDR, is anticipated in the overall cost of the project.

3.3 Tracking Requirements Volatility 

It is important to keep track of requirements changes, especially later in the development lifecycle, so that an assessment can be made of the impact of these changes on the overall project cost and schedule. This assessment allows the project manager to make better-informed decisions on the implementation of the changes. Tracking requirements volatility over time can help identify potential process improvements that need to be made to enhance efficiency and productivity in the future.

3.4 Reporting Requirements Volatility Metrics

Requirements volatility metrics are reported to the project manager and others regularly. They are also reported to the Software Engineering Process Group (SEPG) to help with future project cost estimation.

Some metrics to consider are:

• Monitor and control changes in the software scope.
  • Number of requirements
  • Number of requirements changes
  • Number of TBR/TBDs over time
• Assess the extent and cost of product rework.
  • Rework trend over time
  • Cost of the rework

See also SWE-094 - Reporting of Measurement Analysis. 

3.5 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook:

3.6 Center Process Asset Libraries

See the following link(s) in SPAN for process assets from contributing Centers (NASA Only). 

4. Small Projects

No additional guidance is available for small projects.

5. Resources

5.1 References

5.2 Tools

6. Lessons Learned

6.1 NASA Lessons Learned

No Lessons Learned have currently been identified for this requirement.

6.2 Other Lessons Learned

No other Lessons Learned have currently been identified for this requirement.

7. Software Assurance

7.1 Tasking for Software Assurance

7.2 Software Assurance Products.

• SA analysis of software volatility measures over time. 

  
  

  Objective Evidence

  • Software measurement or metric data showing software volatility
  • Trends and analysis results on the metric set being provided
  • Status presentation showing metrics and treading data
   Definition of objective evidence

  Objective evidence is an unbiased, documented fact showing that an activity was confirmed or performed by the software assurance/safety person(s). The evidence for confirmation of the activity can take any number of different forms, depending on the activity in the task. Examples are:

  • Observations, findings, issues, risks found by the SA/safety person and may be expressed in an audit or checklist record, email, memo or entry into a tracking system (e.g. Risk Log).
  • Meeting minutes with attendance lists or SA meeting notes or assessments of the activities and recorded in the project repository.
  • Status report, email or memo containing statements that confirmation has been performed with date (a checklist of confirmations could be used to record when each confirmation has been done!).
  • Signatures on SA reviewed or witnessed products or activities, or
  • Status report, email or memo containing a short summary of information gained by performing the activity. Some examples of using a “short summary” as objective evidence of a confirmation are:
    • To confirm that: “IV&V Program Execution exists”, the summary might be: IV&V Plan is in draft state. It is expected to be complete by (some date).
    • To confirm that: “Traceability between software requirements and hazards with SW contributions exists”, the summary might be x% of the hazards with software contributions are traced to the requirements.
  • The specific products listed in the Introduction of 8.16 are also objective evidence as well as the examples listed above.

7.3 Metrics 

• Software Requirements Volatility (# of requirements added, deleted, modified, # of TBDs over time)

       Note: Metrics in bold type are required for all projects

See also Topic 8.18 - SA Suggested Metrics.

7.4 Guidance 

Requirements volatility is the change in requirements (added, deleted, and modified) over a given time interval. The success of software projects is dependent on the quality of requirements, especially given that requirements are the basis for planning project schedules as well as design and test specifications.

Examples:

Software requirements volatility is expected during the early stages of a project (conceptualize / requirements phase). It becomes a concern when it occurs after the software requirements phase (after CDR) is complete because it is likely to result in the re-work of the software components. 

Even though there are good requirements, in the beginning, the software requirements may be changed during the project development. Software requirement changes during the software development process are also known as Requirements Volatility.

Requirements volatility has a great impact on the cost, the schedule, and the quality of the final product. Due to requirements volatility, many projects fail and some are completed partially. Requirements volatility cannot be overcome fully but we can minimize it with some requirement measures or metrics. Requirement metrics are useful in identifying the risks of a project by locating errors in the requirements document.

Requirements evolution is due to factors.

1. External Factors:
   1. Government regulations
   2. project direction and changes
   3. project funding
2. Internal factors
   1. Hardware and interface constraints and unknowns 
   2. Lack of experience in the system requirements and software requirements development team
   3. Feedback from milestone reviews and peer reviews
   4. Complexity 
   5. Customer, operational, software, and hardware changes
   6. Hazard identifications
   7. Lack of process maturity in requirements generation and requirements management
   8. Requirement or hardware reuse
   9. Environmental changes or mission profile changes
3. Requirements instability (the extent of fluctuation in user requirements)
4. Requirements diversity (the extent to which stakeholders disagree among themselves deciding on requirements).
5. Requirements analyzability (the extent to which the process of producing requirement -specification can be reduced to objective procedure).
6. Poor communication between users and the development team.

Requirements volatility contains schedule, cost, and performance risk factors. Developing and maintaining a trending of software requirements volatility measurements is critical to determine the risks associated with the project.

Requirement volatility is not the only reason contributing to the success of any project, by can be a key indicator of the project risk and success.

Several challenges lead to Requirement Volatility.

1. The change request form was not always complete. A change request form may have little information about the reason for the proposed change.
2. No formal impact analysis and incomplete change effort estimation.
3. Traceability between requirements and other software artifacts is not established. See SWE-052 - Bidirectional Traceability

The impact of requirement volatility on the software project has been seen as:

1. Projects Schedule: If the schedule of one activity is delayed, obviously all subsequent activities schedule will be disturbed.
2. Project Performance: Project performance decreases due to the changes in requirements. Requirement Volatility has a high impact on the coding and maintenance phases.
3. Project Cost: Project cost increases due to a change in requirements.
4. S/W Maintenance
5. S/W Quality: The quality of software decreases due to continuous changes in requirements.

Though requirements volatility has an impact on the project schedule, project performance, project cost, software maintenance, and software quality, it may have some positive effects as well as may help us to have a better understanding of user requirements.

7.5 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook: