See edit history of this section
Post feedback on this section
- 1. The Requirement
- 2. Rationale
- 3. Guidance
- 4. Small Projects
- 5. Resources
- 6. Lessons Learned
- 7. Software Assurance
1. Requirements
3.6.1 The project manager shall plan and implement software assurance, software safety, and IV&V (if required) per NASA-STD-8739.8, Software Assurance and Software Safety Standard.
1.1 Notes
Software assurance activities occur throughout the life of the project. Some of the actual analyses and activities may be performed by engineering or the project. Software Assurance directions, requirements, and guidance can be found in the NASA-STD-8739.8.
1.2 History
1.3 Applicability Across Classes
Class A B C D E F Applicable?
Key: - Applicable | - Not Applicable
2. Rationale
The software assurance and software safety activities provide a level of confidence that software is free from vulnerabilities, either intentionally designed into the software or accidentally inserted at any time during its life cycle, that the software functions in an intended manner, and that the software does not function in an unintended manner. The software assurance process is the planned and systematic set of activities that ensure the conformance of software life cycle processes and products to requirements, standards, and procedures. Software assurance assures that the software and its related products meet their specified requirements, conform to standards and regulations, are consistent, complete, correct, safe, secure, and reliable as warranted for the system and operating environment, and satisfy customer needs. The objectives of software assurance and software safety activities include the following:
- Ensuring that the processes, procedures, and products used to produce and sustain the software conform to all specified requirements and standards that govern those processes, procedures, and products.
- A set of activities that assess adherence to, and the adequacy of the software processes used to develop and modify software products.
- A set of activities that define and assess the adequacy of software processes to provide evidence that establishes confidence that the software processes are appropriate for and produce software products of suitable quality for their intended purposes.
- Determining the degree of software quality obtained by the software products.
- Ensuring that the software systems are safe and that the software safety-critical requirements are followed.
- Ensuring that the software systems are secure.
3. Guidance
3.1 Software Assurance and Software Safety Standard
The purpose of the Software Assurance and Software Safety Standard is to define the requirements to implement a systematic approach to Software Assurance (SA), Software Safety, and IV&V for software created, acquired, provided, or maintained by or for NASA. Various personnel in the program, project, or facility, and Safety and Mission Assurance (SMA) organizations can perform the activities required to satisfy these requirements. The Software Assurance and Software Safety Standard provides a basis for personnel to perform software assurance, software safety, and IV&V activities consistently throughout the life of the software, that is, from its conception, through creation to operations and maintenance, and until the software is retired.
This includes products delivered to and used within NASA, and products developed and acquired by NASA. Software assurance assists in risk mitigation by helping expose potential defects in products and processes, thus preventing problems from evolving. However, it also, through its metrics, tracking, and analysis activities, enables the improvement of future products and services. Software assurance often serves as the corporate memory from project to project, sharing potential problem areas and lessons learned.
3.2 Objectives of Software Assurance
Software assurance is the level of confidence that software is free from vulnerabilities, either intentionally designed into the software or accidentally inserted at any time during its life cycle, that the software functions in an intended manner, and that the software does not function in an unintended manner. The objectives of the Software Assurance and Software Safety Standard include:
a. Ensuring that the processes, procedures, and products used to produce and sustain the software conform to all requirements and standards specified to govern those processes, procedures, and products.
b. Ensuring that the software systems are safe and that the software safety-critical requirements and processes are followed.
c. Ensuring that the software systems are secure.
3.3 Application and Approach
The application and approach to meeting the Software Assurance and Software Safety activities will vary based on the system and software products and processes to which they are applied. The Software Assurance and Software Safety Standard stresses coordination between the software assurance sub-disciplines, as well as with system safety, system reliability, hardware quality, system security, and software engineering, to maintain the system perspective and minimize duplication of effort.
Project and SMA Management support of the Software Assurance function is essential for Software Assurance processes to be effective. The Software Assurance support minimally includes:
(1) Management is familiar with and understands the Software Assurance function’s purposes, concepts, practices, and needs.
(2) Management provides the Software Assurance function with an appropriate level of skilled resources (people, equipment, knowledge, methods, facilities, and tools) to accomplish their project responsibilities.
(3) Management receives and acts upon information provided by the Software Assurance function throughout a project.
3.4 Responsibilities
Software engineering and the software assurance disciplines are integrally related and yet each has its responsibilities. Jointly they are responsible for providing project management with the optimal solution for software to meet the engineering, safety, quality, and reliability needs of the project. This necessitates a close working relationship to establish the appropriate levels of effort for both. The NASA Procedural Requirements, NPR 7150.2 083, NASA Software Requirements invokes the NASA Software Assurance and Software Safety Standard (NASA-STD-8739.8 278), requiring a close working relationship, understanding of roles and responsibilities, and establishing expected communication paths. NPR 7150.2, besides laying out the NASA minimum requirements for software development, provides the NASA software classification upon which software engineering, software assurance, and software safety all base their tailoring.
Software assurance includes the disciplines of software assurance, software quality, software safety, and Independent Verification & Verification (IV&V). All of these activities work together to ensure high-quality products that operate safely. The amount of software assurance required for a project is based on the classification of the software SWE-020 - Software Classification and the risk associated with the project and the software.
3.5 NASA's Safety and Mission Assurance (SMA)
NASA's Safety and Mission Assurance organizations assure that the products and processes are implemented according to the agreed-upon plan(s). Software assurance is recommended for software activities and products, including solicitations, contracts, and memorandums of agreements, software plans, requirements, design, implementation, verification, validation, certification, acceptance, maintenance, operations, and retirement activities. See also Topic 8.51 - Software Assurance Plan,
To avoid misunderstandings and issues once the project has begun, the following software assurance items need to be considered for inclusion in the provider's contract (Topic 7.03 - Acquisition Guidance):
- For access to provider processes and documentation, see SWE-040 - Access to Software Products.
- Software assurance activities, meetings, reviews, etc. are planned for the project.
- Standards and requirements for provider-performed software assurance activities.
- Software assurance metrics are to be collected and reported by the software provider.
Use checklists, documented procedures, templates, etc. to ensure consistency among software assurance activities across projects.
See also Topic 5.05 - Metrics - Software Metrics Report,
3.6 SA Reviews and Audits
The team is to implement the software engineering products and processes according to the plans established by the project. Software assurance activities ensure this happens through a series of reviews and audits. For NASA, NASA-STD-8739.8 captures the software assurance and software safety requirements in a single place for more uniform applications across projects where software is developed or acquired for NASA. See also Topic 8.12 - Basics of Software Auditing.
See also SWE-045 - Project Participation in Audits,
3.7 Additional Guidance
Additional guidance related to this requirement may be found in the following materials in this Handbook:
Related Links |
---|
3.8 Center Process Asset Libraries
SPAN - Software Processes Across NASA
SPAN contains links to Center managed Process Asset Libraries. Consult these Process Asset Libraries (PALs) for Center-specific guidance including processes, forms, checklists, training, and templates related to Software Development. See SPAN in the Software Engineering Community of NEN. Available to NASA only. https://nen.nasa.gov/web/software/wiki 197
See the following link(s) in SPAN for process assets from contributing Centers (NASA Only).
SPAN Links |
---|
4. Small Projects
Software assurance is required regardless of project size. Since NASA-STD-8739.8 278 defines roles, not individuals, projects with limited personnel resources, can use one person to fulfill multiple roles and perform multiple software assurance functions, as long as the proper independence for the specific requirement is maintained. In small project situations, sometimes it will be necessary for a project to have software assurance conducted by someone not on the project, but potentially from the same organization. This is not the preferred approach, but better than having no software assurance done at all.
Additionally, for acquired software, software assurance is performed by both the acquirer and the provider, so projects with small acquirer staff could consider doing more "insight" (Surveillance mode requiring the monitoring of customer identified metrics and contracted milestones) per NASA-STD-8739.8, "Tailoring the implementation of software assurance requirements is acceptable commensurate with the program/project classification as well as size, complexity, criticality, and risk."
5. Resources
5.1 References
- (SWEREF-033) SMA-SA-WBT-100 SATERN. System for Administration, Training, and Educational Resources for NASA (SATERN), accessible to NASA-users at https://saterninfo.nasa.gov/. User needs account to access SATERN courses. This NASA-specific information and resource is available in at the System for Administration, Training, and Educational Resources for NASA (SATERN), accessible to NASA-users at https://saterninfo.nasa.gov/.
- (SWEREF-050) SMA-SA-WBT-203 SATERN. System for Administration, Training, and Educational Resources for NASA (SATERN), accessible to NASA-users at https://satern.nasa.gov/. User needs account to access SATERN courses. This NASA-specific information and resource is available in at the System for Administration, Training, and Educational Resources for NASA (SATERN), accessible to NASA-users at https://satern.nasa.gov/.
- (SWEREF-055) SMA-SA-WBT-320 SATERN. System for Administration, Training, and Educational Resources for NASA (SATERN), accessible to NASA-users at https://satern.nasa.gov/. User needs account to access SATERN courses. This NASA-specific information and resource is available in at the System for Administration, Training, and Educational Resources for NASA (SATERN), accessible to NASA-users at https://satern.nasa.gov/.
- (SWEREF-083) NPR 7150.2D, Effective Date: March 08, 2022, Expiration Date: March 08, 2027 https://nodis3.gsfc.nasa.gov/displayDir.cfm?t=NPR&c=7150&s=2D Contains link to full text copy in PDF format. Search for "SWEREF-083" for links to old NPR7150.2 copies.
- (SWEREF-197) Software Processes Across NASA (SPAN) web site in NEN SPAN is a compendium of Processes, Procedures, Job Aids, Examples and other recommended best practices.
- (SWEREF-270) OSMA Website
- (SWEREF-271) NASA STD 8719.13 (Rev C ) , Document Date: 2013-05-07
- (SWEREF-278) NASA-STD-8739.8B , NASA TECHNICAL STANDARD, Approved 2022-09-08 Superseding "NASA-STD-8739.8A,
- (SWEREF-433) Rosenberg, Dr. Linda H. NASA Goddard Space Flight Center (GSFC). In Journal of Software Technology, Vol. 6, Number 2. October, 2003. Lessons Learned Reference.
- (SWEREF-444) Rosenberg, Linda H. (Oct, 2003). Journal of Software Technology.
- (SWEREF-468) Wetherholt, Martha. CrossTalk - Journal of Defense Software Engineering, Sep/Oct, 2015. Retrieved October 08, 2015 from http://www.crosstalkonline.org/storage/flipbooks/2015/201509/index.html. See page 31.
5.2 Tools
NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN.
The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool. The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.
6. Lessons Learned
6.1 NASA Lessons Learned
No Lessons Learned have currently been identified for this requirement.
6.2 Other Lessons Learned
- Software quality assurance implementation is a balancing activity that must be tailored as project appropriate 444: (Lesson 2 of 12)
Software Quality Assurance engineers must determine, based on experience, what trade-offs are to be made within the specific project since each project has different objectives. Gilles defined some of these relationships between the quality assurance criteria e.g., an inverse relationship between usability and efficiency; a direct relationship between maintainability and reusability 2. To make the best trade-offs, all relationships must be understood and experience used to interpret the impact."
- Software quality assurance (SQA) must evaluate the process as well as the products 444: (Lesson 3 of 12)
Within NASA, SA tends to focus on the products produced, such as the requirements documents, design, code listing, and test plans. But the focus of SA is to monitor continuously throughout the Software Development Life Cycle to ensure the quality of the delivered products. This requires monitoring both the processes and the products. In process assurance, SA provides management with objective feedback regarding compliance with approved plans, procedures, standards, and analyses. Product assurance activities focus on the changing level of product quality within each phase of the life cycle, such as the requirements, design, code, and test plan. The objective is to identify and eliminate defects throughout the life cycle as early as possible, thus reducing test and maintenance costs.
- There must be a software assurance plan 444: (Lesson 4 of 12)
The plan will specify the goals, what is to be performed, standards against which the development work is to be measured, all relevant procedures, and the organizational structure of the Quality Assurance within the project.
- Software quality assurance must span the entire software development life cycle 444: (Lesson 5 of 12)
Software Development starts at the concept phase and continues through maintenance. SA activities and funding should also start at the concept definition and continue through the entire life cycle. At each phase, some activities could be performed 4. The project and the quality assurance office must work together to negotiate what activities are appropriate at each phase based on risks and resources.
- Requirements are the birthplace of successful projects 444: (Lesson 6 of 12)
Although SA is performed across the entire life cycle, the success of a project can often be determined by the attention paid to requirements. The earlier in the life cycle potential risks are identified, the easier it is to eliminate or at least manage the conditions that introduce that risk. Problems not found until the testing phase are approximately 14 times more costly to fix than if found and fixed earlier in the requirements phase. The first tangible representation of the capabilities produced is the requirements specification document, be it system, hardware, software, or operational requirements. The document also serves to establish the basis for all the project's engineering management and assurance functions. If the quality of the requirements specification is poor, the project is at risk even before work begins.7 Therefore, a specific lesson in SA is the emphasis on requirements.
- Software quality assurance does not equal testing 444: (Lesson 7 of 12)
From the aspect of quality assurance, the purpose of testing is to:
- Assure problems are documented, corrected, and used for process improvement.
- Assure problem reports are assessed for their validity.
- Assure reported problems and their associated corrective actions are implemented following customer-approved solutions.
- Provide feedback to the developer and the user on the problem status.
- Provide data for measuring and predicting Software Quality and Software Reliability."
- Metrics are a necessity 444: (Lesson 8 of 12)
Software Metrics are often ignored during the early Software Development Life Cycle phases and are not actively associated with SA - but should be. For SA practitioners, with their responsibility for assuring both the processes and the products of the Software Development, measurement is critical. Throughout each of the life cycle phases metrics have proven invaluable in the evaluation of the quality of the products and processes.
- Safety and reliability are critical aspects of SQA 444: (Lesson 9 of 12)
Safety is a team effort and everyone's responsibility. Software within NASA is a vital part of the system and can impact the safety of the mission. Project managers, Systems Engineers, Software Leads and engineers, Software Assurance or Quality Assurance, and System Safety Personnel all play a part in creating a safe system. Reliability and safety cannot be tested at the end of a project; they must be built in as the software is being Developed.
Reliability impacts safety - a system cannot be deemed safe if it is not reliable.
- Independent verification and validation (IV&V) is an important tool within SQA 444: (Lesson 10 of 12)
NASA defines Software IV&V as a Systems Engineering Process employing rigorous methodologies for evaluating the correctness and quality of the software product throughout the Software Development Life Cycle.
- Hardware does not equal software 444! (Lesson 11 of 12)
The influence of Hardware Quality Assurance is evident in the Software Quality Assurance practitioner community, where hardware-intensive systems and typical hardware-related concerns predominate. Two issues that dominate discussions about hardware are time and operating conditions. Software, however, is built with different constraints and considerations.
- Risk Management is not optional 444: (Lesson 12 of 12)
Risk is a daily reality in all projects, and Continuous Risk Management should become just as routine. It should be ongoing and comfortable and neither imposed nor forgotten. Like any good habit, it should seamlessly fit into daily work.
Software Risk Management is important because it helps avoid disasters, rework, and overkill. The objectives of Software Risk Management are to identify, address, and eliminate software risk items before they become threats to success or major sources of rework. Good project managers are also good managers of risk. It makes good business sense for all software development projects to incorporate risk management as part of project management.
7. Software Assurance
7.1 Tasking for Software Assurance
7.2 Software Assurance Products
Evidence shows that the project has implemented NASA-STD-8739.8 278.
- Software Assurance Process Audit Report
- Audit results on the requirements of NASA-STD 8739.8, including findings and corrective action plans.
Objective Evidence
- NPR 7150.2 and NASA-STD-8739.8 requirements mapping matrices signed by the engineering and SMA technical authorities for each development organization.
See also Topic 8.59 - Audit Reports,
7.3 Metrics
- # of Non-Conformances per audit (including findings from process and compliance audits, process maturity)
- # of software work product Non-Conformances identified by life cycle phase over time
See also Topic 8.18 - SA Suggested Metrics
7.4 Guidance
Step 1 Determine the Software Assurance plan and the Software Assurance and Software Safety Standard requirements mapping per the software classification and the engineering requirements mapping matrix to verify the methods to fulfill the requirements in NASA-STD-8739.8.
Step 2 Perform an audit, at least every two years or upon request, to verify the software assurance and software safety requirements are being followed and that the products and metrics are being produced and provided to the engineering and project office.
Step 3 Monitor issues, corrective actions, and findings to closure- assess the engineering, contractor, software assurance, and software safety issues, corrective actions, and finding to determine if the items have been addressed.
The software assurance process is the planned and systematic set of activities that ensure the conformance of software life cycle processes and products to requirements, standards, and procedures. Software assurance assures that the software and its related products meet their specified requirements, conform to standards and regulations, are consistent, complete, correct, safe, secure, and reliable as warranted for the system and operating environment, and satisfy customer needs.
- Assess the Software Assurance plan and the Software Assurance and Software Safety Standard requirements mapping to verify the methods to fulfill the requirements in NASA-STD-8739.8.
Software requirements tailoring is the process used to seek relief from standard requirements consistent with program or project objectives, acceptable risk, and constraints. To accommodate the wide variety of software systems and subsystems, application of these requirements to specific software assurance, software safety, and IV&V efforts may be tailored where justified and approved.
NASA established the Technical Authority governance model to approve and mitigate any changes to the application of the requirements in the Software Assurance and Software Safety Standard. Tailoring from requirements in the Software Assurance and Software Safety Standard is governed by the following requirements, as well as those established in NPR 8715.3 for all of the Agency’s investment areas. The responsible program, project, or operations manager must formally accept the risk. Tailoring at the Center level is to be decided by the SMA Technical Authority. The software assurance, software safety, and IV&V requirements are tailored using the following levels:
- The first level of tailoring is the Software Classification Decision, see NPR 7150.2 083.
- The second level of tailoring is the tailoring in the project’s Software Requirements Mapping Matrix, see NPR 7150.2.
- The third level of tailoring is the tailoring by the Software Assurance TA of the Software Assurance and Software Safety requirements that correspond to the project’s Software Requirements Mapping Matrix requirements.
The Software Assurance and Software Safety Standard establishes a baseline set of requirements to reduce software assurance, software safety, and IV&V risks on NASA projects and programs. Each project has unique circumstances, and tailoring can be employed to modify the requirements set appropriate for software assurance, software safety, and IV&V effort.
Determining the tailoring of requirements is a joint software engineering effort and SMA effort, including acceptable technical and programmatic risk posture, Agency priorities, size, and complexity. Requirements can be tailored more broadly across a group of similar projects, a program, an organization, or another collection of similar software development efforts.
The software assurance organization should maintain a requirements mapping matrix or multiple requirements mapping matrices against requirements in the Software Assurance and Software Safety Standard, including those delegated to other parties or accomplished by contract vehicles.
The software assurance organization will conduct risk assessment efforts to determine the software assurance, software safety, and IV&V tasks to be performed, and the rigor of each task. The software assurance organization will develop, maintain, and execute plans and procedures that cover the entire software life cycle and, as a minimum, address the requirements of the Software Assurance and Software Safety Standard with approved tailoring.
The approval of the Software Assurance and Software Safety Standard tailoring is determined by the appropriate SMA management at the Center Level in conjunction with the project.
The request for relief from a requirement includes:
- the rationale,
- a risk evaluation, and
- reference to all material that justifies supporting acceptance.
The organization submitting the tailoring request informs the next higher level of involved management promptly of the tailoring request. The dispositioning organization reviews the request with the other organizations that could be impacted or have a potential risk (i.e., to safety, quality, cybersecurity, health) with the proposed changes; and obtains the concurrence of those organizations.
The Center SMA technical authority shall review and agree with any tailored Software Assurance and Software Safety Standard requirements. (SASS-9)
If a system or subsystem development evolves to meet a higher or lower software classification as defined in NPR 7150.2 then the software assurance, software safety, and IV&V organizations shall update their plan(s) to fulfill the applicable requirements per the Requirements Mapping Matrix and any approved changes, and initiate adjustments to applicable contracts to meet the modified requirements. (SASS-10)
- Perform an audit of SA performance against the requirements in SA plans at least every two years or upon request - Plan and conduct an audit to verify that the selected software assurance and software safety requirements are being followed by whoever is responsible for performing the software assurance and software safety activities, the software assurance and software safety products exist, and the software assurance and software safety metrics are being recorded and used.
- Monitor issues, corrective actions, and findings to closure. - identify any issues with meeting and executing the software assurance and safety standard requirements and with the implementation of NPR 7150.2 requirements, identify risks associated with not meeting any of the requirements, and track all issues to closure.
Periodic audits against SA plans and requirements as well as regular status monitoring and daily activities with the engineering teams can help identify requirements that are not being met and the associated risks.
7.5 Additional Guidance
Additional guidance related to this requirement may be found in the following materials in this Handbook:
Related Links |
---|