1. Requirements

2.1.2.3 The NASA Chief, SMA shall periodically benchmark each Center’s software assurance and software safety capabilities against the NASA-STD-8739.8, NASA Software Assurance and Software Safety Standard.

1.1 Notes

NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.

1.2 History

2. Rationale

The Headquarters Office of Safety and Mission Assurance (OSMA) is responsible for ensuring that the Agency-level software assurance and software safety requirements and policies are being followed throughout the Agency.

3. Guidance

3.1 The NASA Organization Agency Assessment Responsibilities

NPD 1000.3E,   ⁰⁶⁶  Section 4.13 Office of Safety and Mission Assurance

The Office of Safety and Mission Assurance provides policy direction, functional oversight, and assessment for all Agency safety, reliability, maintainability, and quality engineering and assurance activities and serves as a principal advisory resource for the Administrator and other senior officials on matters pertaining to safety and mission success.

NPD 1000.3E, Section 5.15 NASA Safety Center

Manages the audit, review, and assessment process for evaluating and ensuring conformance with Agency SMA requirements.

NPD 8700.1E,  ⁰³⁶   NASA Policy for Safety and Mission Success

Verify and validate the life cycle implementation of the SMA processes and any related safety and mission success requirements through ongoing surveillance of program, project, and contractor processes.

NPR 8705.6D ³⁵³, Safety and Mission Assurance (SMA) Audits, Reviews, and Assessments

The NSC AIO conducts audits, reviews, and assessments to verify NASA program and project implementation of, and compliance with, applicable Agency SMA requirements.

The Headquarters Office of Safety and Mission Assurance (OSMA) achieves this requirement by a number of methods:

• NASA Quality Audit, Assessment, and Review (QAAR) audit, periodic assessment of compliance at the Center and within the programs/projects to verify that they are meeting this responsibility.
• Review of the Capability Maturity Model Integration (CMMI®) appraisal results.
• Review and participate in program and project reviews.
• Review of Center and project planning documents, schedule, and progress.
• Review of Center and project waivers.
• Feedback and status presentations provided by the Centers during the NASA Software Assurance Working Group activities.
• Project status and feedback provided to the NASA Headquarters OSMA.
• Software inventory data.
• External Agency inquires.

The Headquarters Office of Safety and Mission Assurance (OSMA) performs Center and organizational surveys. These surveys are used by the OSMA to provide oversight, maintain internal control, review its operations, and assess compliance with Agency policy.  The OSMA appraisal process addresses several objectives. They are:

• Review Center and specified NASA Headquarters organizations' processes and infrastructure for compliance with OSMA requirements and policies.
• Review specific program/project "files" for compliance with requirements and policies.
• Identify systemic problems or deficiencies.
• Recognize areas of excellence/best practices.
• Receive Center feedback regarding areas where Agency policy and requirements may need to be modified.

See also Topic 8.12 - Basics of Software Auditing

3.2 Audit scope is the software implementation approach and identification of any risks in the following two areas:

Software Assurance and Software Safety Standard requirements, NASA-STD-8739.8A ²⁷⁸

NASA Software Engineering Requirements, NPR 7150.2D ⁰⁸³

3.3 Audit Focus Areas includes:

• Compliance with Agency-level software engineering requirements and policies.
• New standard requirements, including safety-critical software requirements and determination
• Software assurance\safety requirements mapping matrix, review any tailored requirements
• NPR 7150.2D requirements mapping matrix, review any tailored requirements
• Software assurance and safety requirements analysis approach and activities
• Software assurance\safety approach, plan and resource allocations
• Software assurance process audits
• Metric and status reporting by software assurance\safety or planned by software assurance\safety
• Software assurance\safety access to software products and data
• Flow down and implementation approach for the mission Cybersecurity requirements (focus on NPR 7150.2)
• Use of and planned use of Coding standards
• Use of and planned use of tools
• IV&V plan and communication, access to data, project’s interaction with IV&V, IV&V interaction with the project
• Software quality assessment approaches for the project
• Software risks, or known issues
• Software hazards
• Integrated testing approach and plans
• Software engineering and software assurance\safety requirements flow down into contracts
• Open-source software and reused software approach and plans
• Software engineering and software assurance document management system
• Software technical authority implementation at a Center.

See also SWE-036 - Software Process Determination. 

While The Headquarters Office of Safety and Mission Assurance (OSMA) assesses overall Center response to the requirements of the NPR 7150.2, the Capability Maturity Model Integration ( CMMI®) for Development (CMMI-DEV)  ¹⁵⁷ appraisals objectively benchmark the actual progress the Center makes toward software engineering process improvements.  These CMMI appraisals are the preferred benchmarks for objectively measuring progress.

The CMMI-DEV benchmarking activities will evaluate the Center's current and, with follow-on evaluations, improved capabilities in the specific and general practices of software engineering.  The CMMI requirement is a qualifying requirement.  The requirement is included to make sure NASA projects are supported by software development organization(s) having the necessary skills and processes in place to produce reliable products within cost and schedule estimates. This requirement provides NASA with a methodology to measure software development organizations against an industry-wide set of best practices that address software development and maintenance activities applied to products and services. The CMMI is a yardstick against which the maturity of an organization's product development and acquisition processes can be measured and compared with the industry state of the practice. The CMMI requirement provides NASA with an industry-standard approach to help measure and ensure compliance with the intent of the NPR 7150.2 process-related requirements.  This requirement provides NASA with a common methodology to assess internal and external software development organization's processes and helps NASA identify potential risk areas within a given organization's software development processes.  See SWE-032 - CMMI Levels for Class A and B Software for rating requirements and the CMMI material on the Software Engineering Institute's website, which describes the current CMMI model that is used in the evaluation of a Center's software development capabilities.

3.4 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook:

3.5 Center Process Asset Libraries

See the following link(s) in SPAN for process assets from contributing Centers (NASA Only). 

4. Small Projects

No additional guidance is available for small projects.

5. Resources

5.1 References

5.2 Tools

6. Lessons Learned

6.1 NASA Lessons Learned

No Lessons Learned have currently been identified for this requirement.

6.2 Other Lessons Learned

No other Lessons Learned have currently been identified for this requirement.