1. Requirements

4.4.1 The project shall establish and document specific measurement objectives for their project.

1.1 Notes

NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.

1.2 Applicability Across Classes

This requirement has modified applicability for the following classes:

• Classes D-E and Safety Critical are labeled "X +SO." This means that this requirement applies to the safety-critical aspects of the software.
• Class F is labeled with "X (not OTS)." This means that this requirement does not apply to off-the-shelf (OTS) software for these classes.
• Class G is labeled with "P (Center)." This means that a Center defined process which meets a non-empty subset of this full requirement can be used to meet the intent of this requirement.

2. Rationale

Measurement objectives document the reasons for doing software measurement and the accompanying analysis. They also provide a view into the types of actions or decisions that may be made based on results of analysis. Before implementing a measurement program or choosing measures for a project, it is important to know how the measures are going to be used. Establishing measurement objectives in the early planning stages helps ensure organizational and project information needs are being met and that the measures that are selected for collection will be useful.

There is some cost to collecting and analyzing software measures, so it is desirable to keep the measurement set to the minimum that will satisfy information needs. When measures are tied to objectives, a minimum useful set can be easily selected. Defining measurement objectives helps to select and  prioritize the candidate measures to be collected. If a measurement isn't tied to an objective, it probably doesn't need to be collected.

3. Guidance

Measurement objectives document the reasons why software measurement and analysis are performed. Measurement objectives consider both the perspectives of the organization as well as  the project. The sources for measurement objectives may be management, technical, project, product or process implementation needs, such as:  

• Deliver software by the scheduled date.
• Improve cost estimation.
• Complete projects within budget.
• Devote adequate resources to each process area.

Corresponding measurement objectives for these needs might be:

• Measure project progress to ensure it is adequate to achieve completion by the scheduled date.
• Measure the actual project planning parameters against the estimated planning parameters to identify deviations.
• Track the project cost and effort to ensure project completion within budget.
• Measure resources devoted to each process area to ensure they are sufficient.

According to NPR 7150.2, NASA's software measurement programs are designed to meet the following high-level goals:

• To improve future planning and cost estimation.
• To provide realistic data for progress tracking.
• To provide indicators of software quality.
• To provide baseline information for future process improvement.

Centers may further refine these objectives or add some Center-specific objectives. The measurement objectives a project defines are to be based on their Center's objectives and NASA's objectives. Usually project objectives are focused on the information needs the project has to provide information for managing and controlling the project. When establishing measurement objectives, ask what questions will be answered with the data, why you are measuring something and what types of decisions will be made with the data. A project may also have additional objectives to provide information to their Center or to NASA. This information may be used for process improvement or for developing the organizational baselines and trends. For example, Goddard Space Flight Center (GSFC) has defined two high-level objectives for its software projects as follows:

• To assure that the effort is on track for delivery of the required functionality on time and within budget.
• To improve software development practices both in response to immediate customer issues and in support of the organizational process improvement goals.

As you can see, one of these objectives focuses on the project's ability to manage and control the project with quantitative data and the other objective focuses on organizational process improvement.

In addition, GSFC's Measurement Planning Table Tool found in the Tools listing in the Resources section of this SWE, lists the following more specific measurement objectives for their projects:

• Ensure schedule progress is within acceptable bounds.
• Ensure project effort and costs remain within acceptable bounds.
• Ensure project issues are identified and resolved in a timely manner.
• Deliver the required functionality.
• Ensure the performance measures are within margins.
• Ensure that the system delivered to operations has no critical or moderate severity errors.
• Minimize the amount of rework due to defects occurring during development.
• Ensure requirement are complete and stable enough to continue work without undue risk.
• Support future process improvement.

4. Small Projects

Since small projects are typically constrained by budget and staff, they choose the objectives most important to them to help keep the cost and effort within budget. Some Centers have tailored measurement requirements for small projects. Be sure to check your Center's requirements when choosing objectives and associated measures.

Often small projects have difficulty affording  tools that would enable automatic measurement collection. There are several solutions for this issue. Some Centers, such as GSFC, have developed simple tools (often Excel-based) that will produce the measurements automatically. GSFC examples are the staffing tool, the requirements metrics tool, the action item tool, the risk tool, and the problem reporting tool. More information about these tools can be found in the Tools section under the Resources tab in this SWE.

Other solutions for small projects involve organizational support. Some organizations support a measurement person on staff to assist the small projects with measurement collection and some Centers use tools that can be shared by the small projects.

5. Resources

5.1 Tools

Tools to aid in compliance with this SWE, if any, may be found in the Tools Library in the NASA Engineering Network (NEN).

NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN.

The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool. The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.

6. Lessons Learned

The NASA Lesson Learned database contains lessons learned addressing topics that should be kept in mind when planning a software measurement program. The lessons learned below are applicable when choosing objectives and related measures:  

• Know How Your Software Measurement Data Will Be Used. Lesson Number 1772: When software measurement data used to support cost estimates is provided to NASA by a project without an understanding of how NASA will apply the data, discrepancies may produce erroneous cost estimates that disrupt the process of project assessment and approval. Major flight projects should verify how NASA plans to interpret such data and use it in their parametric cost estimating model, and consider duplicating the NASA process using the same or a similar model prior to submission. ⁵⁶⁷

• Space Shuttle Processing and Operations Workforce. Lesson Number 1042: Operations and processing in accordance with the Shuttle Processing Contract (SPC) have been satisfactory. Nevertheless, lingering concerns include: the danger of not keeping foremost the overarching goal of safety before schedule before cost; the tendency in a success-oriented environment to overlook the need for continued fostering of frank and open discussion; the press of budget inhibiting the maintenance of a well-trained NASA presence on the work floor; and the difficulty of a continued cooperative search for the most meaningful measures of operations and processing effectiveness. ⁵⁸³
  
  A recommendation from Lesson Number 1042 states "NASA and SPC should continue to search for, develop, test, and establish the most meaningful measures of operations and processing effectiveness possible."