1. Requirements

2.1.1.2 The NASA OCE shall periodically benchmark each Center's software engineering capability against requirements in this directive. 

1.1 Notes

Capability Maturity Model® Integration (CMMI®) for Development (CMMI-DEV) appraisals are the preferred benchmarks for objectively measuring progress toward software engineering process improvement at NASA Centers.

1.2 History

2. Rationale

The Headquarters Office of the Chief Engineer (OCE) is responsible for ensuring that the Agency-level software engineering requirements and policies are being followed throughout the Agency.

3. Guidance

The Headquarters Office of the Chief Engineer (OCE) achieves this requirement by a number of methods:

• OCE project surveys. OCE personnel conduct periodic assessments of compliance at the Centers and within the programs/projects to verify that they are meeting this responsibility.
• Review of the Capability Maturity Model Integration (CMMI®) appraisal results.
• Review and participate in program and project reviews.
• Review of organizational and project planning documents, schedule, and progress.
• Review of Center and project waivers.
• Feedback and status presentations were provided by the Centers during the NASA Software Working Group activities.
• Feedback and discussion for the NASA Software Working Group members.
• Project status and feedback provided to the NASA Headquarters OCE.
• Software inventory data.
• External Agency inquires.

See also SWE-003 - Center Improvement Plans, SWE-036 - Software Process Determination. 

See also SWE-209 - Benchmarking Software Assurance and Software Safety Capabilities

3.1 Survey Objectives and Core Elements

The Headquarters OCE performs project and organizational surveys. These surveys are used by the OCE to provide oversight, maintain internal control, review its operations, and assess compliance with Agency policy.  The OCE appraisal process addresses several objectives. They are:

• Review project and specified NASA Headquarters organizations' processes and infrastructure for compliance with OCE requirements and policies.
• Review specific program/project "files" for compliance with requirements and policies.
• Identify systemic problems or deficiencies.
• Recognize areas of excellence/best practices.
• Receive feedback regarding areas where Agency policy and requirements may need to be modified.

Currently, the OCE software surveys focus on the following core elements:

• Compliance with Agency-level software engineering requirements and policies.
• Software Engineering technical authority implementation on a project.
• Software engineering dissenting opinions and waiver process at a Center.
• Software engineering management.
• Incorporation of software engineering lessons learned.
• Software Documentation and Records.
• Software Classification process and levels.
• Software Engineering Training approach.
• Software Risk Management approach.
• Software Cost Estimates and Resources Allocation.
• Software insight responsibilities.
• Use of software engineering discipline for software associated with Programmable Logic Devices.
• Software Architecture and Detail Design.
• Software Safety and assurance practices.
• Software Inventory data.
• Inter-Center Software Work.
• Use and recording of software metric data.

3.2 CMMI Use In Benchmarking

While SWE-004 is written from the OCE point of view, the requirement also contains an inherent Center role, i.e., participation in the OCE-sponsored bench-marking activities. A Center's support of this requirement can be assessed by considering the extent of its preparations for and involvement in these benchmarking efforts. While the OCE Compliance Survey assesses the project response to the requirements of the NPR 7150.2, the Capability Maturity Model Integration ( CMMI®) for Development (CMMI-DEV) ¹⁵⁷ appraisals objectively benchmark the actual progress the Center makes toward software engineering process improvements.  These CMMI appraisals are the preferred benchmarks for objectively measuring progress.

The CMMI-DEV benchmarking activities will evaluate the Center's current and, with follow-on evaluations, improved capabilities in the specific and general practices of software engineering.  The CMMI requirement is a qualifying requirement.  The requirement is included to make sure NASA projects are supported by software development organization(s) having the necessary skills and processes in place to produce reliable products within cost and schedule estimates. This requirement provides NASA with a methodology to measure software development organizations against an industry-wide set of best practices that address software development and maintenance activities applied to products and services. The CMMI is a yardstick against which the maturity of an organization's product development and acquisition processes can be measured and compared with the industry state of the practice. The CMMI requirement provides NASA with an industry-standard approach to help measure and ensure compliance with the intent of the NPR 7150.2 ⁰⁸³ process-related requirements.  This requirement provides NASA with a common methodology to assess internal and external software development organization processes and helps NASA identify potential risk areas within a given organization's software development processes.  See SWE-032 - CMMI Levels for Class A and B Software for rating requirements and the CMMI material on the Software Engineering Institute's website, ¹⁵⁷ which describes the current CMMI model that is used in the evaluation of a Center's software development capabilities.

See also SWE-129 - OCE NPR Appraisals, and SWE-221 - OSMA NPR Appraisals for information on NPR appraisals. 

3.3 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook:

3.4 Center Process Asset Libraries

See the following link(s) in SPAN for process assets from contributing Centers (NASA Only). 

4. Small Projects

Typically the OCE includes a small project in the survey activities at a Center. The OCE survey leader will work with the Center SW point of contact (POC) to develop the appropriate level of survey involvement for small projects.

5. Resources

5.1 References

5.2 Tools

6. Lessons Learned

6.1 NASA Lessons Learned

No Lessons Learned have currently been identified for this requirement.

6.2 Other Lessons Learned

No other Lessons Learned have currently been identified for this requirement.