Appendix A. Definitions

Accredit. The official acceptance of a software development tool, model, or simulation (including associated data) to use for a specific purpose.

Analysis. The post-processing or interpretation of the individual values, arrays, files of data, or execution information. It is a careful study of something to learn about its parts, what they do, and how they are related to each other.

Assure. When personnel makes certain that the specified software engineering, software management, and software assurance activities have been performed by others.

Bidirectional Traceability. Association among two or more logical entities that are discernible in either direction (to and from an entity). (ISO/IEC/IEEE 24765, Systems and software engineering – Vocabulary)

Code coverage. The percentage of the software that has been executed (covered) by the test suite.

Commercial Off-the-Shelf Software (COTS). The software product is available for purchase and use without the need to conduct development activities. COTS solutions, as opposed to custom-developed solutions, are typically readily available in the commercial marketplace and ready for use as purchased. 

Computer. A functional unit that can perform substantial computations, including numerous arithmetic operations and logic operations. 

Computer Software Configuration Item. An aggregation of software that is designated for configuration management and treated as a single entity in the configuration management process.

Computer System. A system containing one or more computers and associated software. (Source: ISO/IEC/IEEE 24765)

Condition. (1) measurable qualitative or quantitative attribute that is stipulated for a requirement and that indicates a circumstance or event under which a requirement applies (Systems and software engineering--Systems and software assurance--Part 1: Concepts and vocabulary ISO/IEC/IEEE 15026-1:2019, 3.1.5) (2) description of a contingency to be considered in the representation of a problem, or a reference to other procedures to be considered as part of the condition (Information processing -- Specification of single-hit decision tables, ISO 5806:1984, 3.6) (3) Boolean expression containing no Boolean operators (Software and systems engineering -- Software testing -- Part 4: Test techniques, ISO/IEC/IEEE 29119-4:2015, 4.6)

Contracted Software. Software created for a project by a contractor or subcontractor.

Cybersecurity. The protection of information and information systems from unauthorized access, use, disclosure, disruption, modification, or destruction in order to provide confidentiality, integrity, and availability.

Cyclomatic Complexity. Cyclomatic complexity is a software metric used to indicate the complexity of a program. It is a quantitative measure of the number of linearly independent paths through a function’s source code.

Data. Information for computer processing (e.g., numbers, text, images, and sounds in a form that is suitable for storage in or processing by a computer).

Defect. Any occurrence in a software product that is determined to be incomplete or incorrect relative to the software requirements, expectations for the software, and/or program standards. (Source: NASA-STD-8739.9)

Embedded Computer System. A computer system that is part of a larger system and performs some of the requirements of that system. (Source: ISO/IEC/IEEE 24765) 

Embedded Software. Software that is part of a larger system and performs some of the requirements of that system. (Source: ISO/IEC/IEEE 24765)

Ensure. To secure or guarantee, to make sure or certain.

Establish and Maintain. Formulation, documentation, use/deployment, and current maintenance of the object (usually a document, requirement, process, or policy) by the responsible project, organization, or individual.

Failure. The behavior of the software or system component when a fault is encountered, producing an incorrect or undesired effect of a specified severity. (Source: NASA-STD-8739.9)

Fault. The manifestation of an error in software that may cause a failure. (Source: NASA-STD-8719.24 Annex)

Freeware. Software that is proprietary and that is available for use at no monetary cost. In other words, freeware may be used without payment but may usually not be modified, re-distributed, or reverse-engineered without the author's permission.

Glueware. Software created to connect the off-the-shelf software/reused software with the rest of the system. It may take the form of software that modifies interfaces or add missing functionality, "firewalls" that isolate the off-the-shelf software, or software that check inputs and outputs to the off-the-shelf software and may modify to prevent failures.

Government Off-the-Shelf Software. Government Off-the-Shelf Software refers to Government-created software, usually from another project. The software was not created by the current developers (see software reuse). Usually, the source code is included and documentation, including test and analysis results, is available; e.g., the Government is responsible for the Government off-the-shelf (GOTS) software to be incorporated into another system. 

Independent Verification and Validation (IV&V). Verification and validation performed by an organization that is technically, managerially, and financially independent of the development organization. (Source: ISO/IEC/IEEE 24765) The NASA requirements for Independent Verification and Validation are defined in the NASA-STD-8739.8. 

Information Technology. Any equipment or interconnected system or subsystem of equipment that is used in the automatic acquisition, storage, manipulation, management, movement, control, display, switching, interchange, transmission, or reception of data or information by an executive Agency.   IT also includes computers, ancillary equipment (including imaging peripherals, input, output, and storage devices necessary for security and surveillance), peripheral equipment designed to be controlled by the central processing unit of a computer; software; firmware; and similar procedures, services (including support services), and related resources, but does not include any equipment acquired by a Federal contractor incidental to a Federal contract. (Source: NPR 7120.7) 

Insight. An element of Government surveillance that monitors contractor compliance using Government-identified metrics and contracted milestones. Insight is a continuum that can range from low intensity such as reviewing quarterly reports to high intensity such as performing surveys and reviews. (Source: NPR 7123.1)

Legacy and Heritage Software. Software products (architecture, code, requirements) written specifically for one project and then, without prior planning during its initial development, found to be useful for other projects. See software reuse.

Major Engineering/Research Facility. Used in this document to show research, development, test, or simulation facilities representing a significant NASA investment (facilities with a Current Replace Value equal to or greater than 50 million dollars) which contains software that supports programs and projects managed under NPR 7120.5, NPR 7120.7, or NPR 7120.8 and that have a Mission Dependency Index value equal to or greater than 70.

MC/DC. Modified condition/decision coverage ( MC/DC) is a code coverage criterion used in software testing . MC/DC requires all of the below during testing: Each condition in a decision is shown to independently affect the outcome of the decision. Independence of a condition is shown by proving that only one condition changes at a time. 

Mission Critical. Item or function that should retain its operational capability to assure no mission failure (i.e., for mission success - meeting all mission objectives and requirements for performance and safety). (Source: NPR 8715.3)

Mission Critical Software. Software that can cause, contribute to, or mitigate the loss of capabilities that are essential to the primary mission objectives. 

Mobile Application. A mobile application is an application built using native code for the device or a software Web application that is distributed through the device specific marketplace. Web applications presented via a mobile browser are not considered mobile applications.

Model. A description or representation of a system, entity, phenomena, or process. (Source: NASA-STD-7009) Only for this document, the term "model" refers to models implemented in software.

Modified Off-the-Shelf Software (MOTS). When COTS or legacy and heritage software is reused, or heritage software is changed, the product is considered "modified." The changes can include all or part of the software products and may involve additions, deletions, and specific alterations. An argument can be made that any alterations to the code and design of an off-the-shelf software component constitute "modification," but the common usage allows for some percentage (less than 5 percent of the code changes) of change before the off-the-shelf software is declared to be modified off-the-shelf (MOTS) software. Modified Off-the-Shelf Software may include the changes to the application shell or glueware to add or protect against certain features and not to the off-the-shelf software system code directly. When less than 30 percent of the existing code changes, the product can be considered "modified." If more than 30 percent of the code changes or if the new code is added, the software should be considered a new software development.

Off-the-Shelf Software. Software not developed in-house or by a contractor for the specific project now underway. The software is developed for a purpose different from the current project. Used in practice as an umbrella for COTS, GOTS, MOTS, OSS, freeware, shareware, trial software, demonstration software, legacy software, heritage software, and reuse software.

Open-Source Software. Software where its human-readable source code is made broadly available without cost under an OSS license, which provides conditions for use, reuse, modification/improvement, and redistribution; and often where the software development, management, and planning is done publicly, or easily observable by an individual or organization not previously connected with its open source project.

Primary Mission Objectives. Outcomes expected to be accomplished, which are closely associated with the reason the mission was proposed, funded, developed, and operated (e.g., objectives related to top-level requirements or their flow down).

Process Asset Library. A collection of process asset holdings that may be used by an organization or project. (Source: CMMI® for Systems Engineering/Software Engineering/Integrated Product and Process Development Supplier Sourcing.)

Program. A strategic investment by a Mission Directorate or Mission Support Office that has a defined architecture and technical approach, requirements, funding level, and a management structure that initiates and directs one or more projects. A program defines a strategic direction that the Agency has identified as critical.

Programmable Logic Device. A semiconductor device based on a matrix of configurable logic blocks connected via a configurable interconnect. The circuitry (combinational/sequential logic, memory/storage, input/output) in a PLD is configured to meet design requirements for a desired application after device manufacturing. 

Project. A specific investment having defined goals, objectives, requirements, life cycle cost, a beginning, and an end. A project yields new or revised products or services that directly address NASA’s strategic needs. They may be performed wholly in-house; by Government, industry, academia partnerships; or through contracts with private industry.

Project Manager. A generic term that represents the position in charge of the project.  A project manager could be designated as a project lead, project principal investigator, project scientist, research director, project executive, or some other term, as defined in the project’s governing document.  A project manager is responsible for the formulation and implementation of the R&T project, per the governing document in coordination with the program manager. (NPR 7120.5 defines the roles and responsibilities for this position). 

Requirements Volatility. The total number of requirements compared to requirement changes over time. It may include additions, changes, and reduction of requirements. 

Risk Management. An organized, systematic decision-making process that efficiently identifies, analyzes, plans, tracks, controls, communicates, and documents risk to increase the likelihood of achieving program/project goals. (Source: NPR 8715.3)

Safety-Critical. A term describing any condition, event, operation, process, equipment, or system that could cause or lead to severe injury, major damage, or mission failure if performed or built improperly, or allowed to remain uncorrected. (Source NPR 8715.3) 

Scripts. A sequence of automated computer commands embedded in a program that tells the program to execute a specific procedure (e.g., files with monitoring, logic, or commands used by software to automate a process or procedure).

Simulation. The imitation of the behavioral characteristics of a system, entity, phenomenon, or process. (Source: NASA-STD-7009) Only for the purpose of this document, the term "simulation" refers to only those simulations that are implemented in software.

Shareware. Software that is available free of charge and often distributed informally for evaluation, after which a fee may be requested for continued use.

Software. In this directive, “software” is defined as (1) computer programs, procedures and associated documentation and data pertaining to the operation of a computer system (IEEE 828-2012, 2.1) (2) all or a part of the programs, procedures, rules, and associated documentation of an information processing system (ISO/IEC 19770-5:2015, Information technology, 3.34) (3) program or set of programs used to run a computer (ISO/IEC 26514:2008, Systems and software engineering–requirements for designers and developers of user documentation, 4.46) (4) all or part of the programs which process or support the processing of digital information (ISO/IEC 19770-1:2017, Information technology – IT asset management – Part 1:  IT asset management systems--Requirements, 3.49) (5) part of a product that is the computer program or the set of computer programs (ISO/IEC/IEEE 26513:2017, Systems and software engineering–requirements for testers and reviewers of information for users, 3.34).  This definition applies to software developed by NASA, software developed for NASA, software maintained by or for NASA, COTS, GOTS, MOTS, OSS, reused software components, auto-generated code, embedded software, the software executed on processors embedded in programmable logic devices (see NASA-HDBK-4008), legacy, heritage, applications, freeware, shareware, trial or demonstration software, and open-source software components.

Software Architecture. The software architecture of a program or computing system is the structure or structures of the system, which comprise software components, the properties of those components, and the relationships between them.  The term also refers to documentation of a system's software architecture.  Documenting software architecture facilitates communication between stakeholders, documents early decisions about high-level design, and allows reuse of design components and patterns between projects.

Software Assurance. The planned and systematic set of activities that ensure that software life cycle processes and products conform to requirements, standards, and procedures. For NASA, this includes the disciplines of Software Quality (functions of Software Quality Engineering, Software Quality Assurance, and Software Quality Control), Software Safety, Software Reliability, Mission Software Cybersecurity Assurance, Software Verification and Validation, and IV&V.

Software Engineering. The application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software, i.e., the application of engineering to software. (Source: ISO/IEC/IEEE 24765)

Software Item. Source code, object code, control code, control data, or a collection of these items.

Software Maintenance. (1) Totality of activities required to provide cost-effective support to a software system. (2) Entitlement of additional rights (such as additional functionality, upgrade, or support) for a previously granted software entitlement.  (3) A set of services a Publisher can sell to a Customer for the ongoing development and delivery of software bug fixes and product upgrades. 

Software Peer Review and Inspection. A visual examination of a software product to detect and identify software anomalies, including errors and deviations from standards and specifications. (Source: IEEE 1028). Refer to NASA-STD-8739.9 for guidelines for software peer reviews or inspections. 

Software Reuse. Software Reuse. A software product developed for one use but having other uses or one developed specifically to be usable on multiple projects or in multiple roles on one project. Examples include, but are not limited to, COTS products, acquirer-furnished software products, software products in reuse libraries, and pre-existing developer software products

Software Suppliers. An organization or individual that enters into an agreement with the acquirer for the supply of a software product or service or individual or organization that enters into a contract with the acquirer for the supply of a software system, software product, or software service under the terms of the contract or an organization or part of an organization or individual that enters into an agreement with the application management organization for the supply of a software product or software service. Software Suppliers includes NASA in-house software development.

Software Validation. Confirmation that the product, as provided (or as it will be provided), fulfills its intended use. In other words, validation ensures that “you built the right thing.” (Source: IEEE 1012, IEEE Standard for Software Verification and Validation)

Software Verification. Confirmation that products properly reflect the requirements specified for them. In other words, verification ensures that “you built it right.” (Source: IEEE 1012) 

Static Analysis. The process of evaluating a system or component based on its form, structure, content, or documentation. (Source: ISO/IEC/IEEE 24765)

Subsystem. A secondary or subordinate system within a larger system. (Source: ISO/IEC/IEEE 24765)

System. The combination of elements that function together to produce the capability required to meet a need. The elements include hardware, software, equipment, facilities, personnel, processes, and procedures needed for this purpose. (Source: NPR 7123.1)

Tailoring. The process used to adjust a prescribed requirement to accommodate the needs of a specific task or activity (e.g., program or project). Tailoring may result in changes, subtractions, or additions to a typical implementation of the requirement.

Uncertainty. (1) The estimated amount or percentage by which an observed or calculated value may differ from the true value. (2) A broad and general term used to describe an imperfect state of knowledge or a variability resulting from a variety of factors including, but not limited to, lack of knowledge, the applicability of information, physical variation, randomness or stochastic behavior, indeterminacy, judgment, and approximation. (Source: NPR 8000.4, Agency Risk Management Procedural Requirements).

Unit Test. (1) Testing of individual routines and modules by the developer or an independent tester (ISO/IEC/IEEE 24765).  (2) A test of individual programs or modules in order to ensure that there are no analysis or programming errors (ISO/IEC 2382-20).  (3) Test of individual hardware or software units or groups of related units. (ISO/IEC/IEEE 24765) 

Validation. (1) Confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled (ISO/IEC 25000:2014 Systems and software Engineering--Systems and software product Quality Requirements and Evaluation (SQuaRE) -- Guide to SQuaRE, 4.41) (ISO/IEC/IEEE 12207:2017 Systems and software engineering--Software life cycle processes, 3.1.71) (ISO/IEC/IEEE 15288:2015 Systems and software engineering--System life cycle processes, 4.1.53) (ISO/IEC TS 24748-1:2016 Systems and software engineering--Life cycle management--Part 1: Guide for life cycle management, 2.61) (2) process of providing evidence that the system, software, or hardware and its associated products satisfy requirements allocated to it at the end of each life cycle activity, solve the right problem (e.g., correctly model physical laws, implement business rules, and use the proper system assumptions), and satisfy intended use and user needs (IEEE 1012-2016, IEEE Standard for Software Verification and Validation, 3.1.35) (3) the assurance that a product, service, or system meets the needs of the customer and other identified stakeholders. It often involves acceptance and suitability with external customers. (A Guide to the Project Management Body of Knowledge (PMBOK(R) Guide) -- Fifth Edition) (4) process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements (IEEE 1012-2016, IEEE Standard for Software Verification and Validation, 3.1) Note: Validation in a system life cycle context is the set of activities ensuring and gaining confidence that a system is able to accomplish its intended use, goals, and objectives. The right system has been built. Validation demonstrates that the system can be used by the users for their specific tasks. "Validated" is used to designate the corresponding status. [ISO 9000] Multiple validations can be carried out if there are different intended uses.

Appendix B. Acronyms

b. CMU/SEI-2010-TR-033 CMMI® for Development, Version 1.3 Software Engineering Institute, Carnegie Mellon University, 2010. See https://cmmiinstitute.com/.

c. DO-178C, Software Considerations in Airborne Systems and Equipment Certification.

d. FAR 52.212-4, Contract Terms and Conditions - Commercial Items.

e. FAR 2005-014, Federal Acquisition Regulation; Smart BUY Pages 61603 – 61605.

f. Federal Information Security Modernization Act of 2014 (FISMA), 44 U.S.C. § 3551, et seq.

g. Federal Information Technology Acquisition Reform Act (FITARA).

h. Government-Wide, US Government Accountability Office, GAO-14-413.

i. IEEE 828-2012, IEEE Standard for Configuration Management in Systems and Software Engineering.

j. IEEE 1012, IEEE Standard for Software Verification and Validation, https://standards.nasa.gov/.

k. IEEE 1028, IEEE Standard for Software Reviews and Audits, https://standards.nasa.gov/.

l. ISO 5806, Information processing -- Specification of single-hit decision tables.

m. ISO/IEC 2382-20, Information technology–Vocabulary–Part 20: System development, 20.05.05.

n. ISO/IEC 19770-1:2017, Information technology – IT asset management – Part 1: IT asset management systems—Requirements.

o. ISO/IEC 19770-5:2015, Information technology.

p. ISO/IEC 26514:2008, Systems and software engineering–requirements for designers and developers of user documentation.

q. ISO/IEC/IEEE 15026-1:2019, Systems and software engineering—Systems and software assurance—Part 1: Concepts and vocabulary.

r. ISO/IEC/IEEE 24765, Systems and software engineering – Vocabulary.

s. ISO/IEC/IEEE 26513:2017, Systems and software engineering–requirements for testers and reviewers of information for users.

t. ISO/IEC/IEEE 29119-4:2015, Software and systems engineering -- Software testing -- Part 4: Test techniques.

u. ISO/IEC 15939, Systems and Software Engineering—Measurement Process.

v. ISO/IEC 19770, International Standards for Software Asset Management Processes.

w. ISO/IEC 24765, Systems and Software Engineering – Vocabulary.

x. NASA-STD-3001, Volumes I-II, Space Flight Human System Standard.

y. NASA-STD-7009, Standard for Models and Simulations, https://standards.nasa.gov/.

z. NASA-STD-8739.9, Software Formal Inspection Standard, https://standards.nasa.gov/.

aa. NASA FAR Supplement 1852.227-86, Commercial Computer Software License in software agreements.

bb. NASA Guidelines for Use of IT Contracts for Supporting End User Services (Update to MFR#137 and MFR#7).

cc. NASA IV&V Management System, https://www.nasa.gov/centers/ivv/ims/home/index.html.

dd. NASA Software Engineering Website, https://nen.nasa.gov/web/software.

ee. NASA Software Process Across NASA (SPAN) Website, https://nen.nasa.gov/web/software/wiki.

ff. NASA/SP-2010-3403, NASA Scheduling Management Handbook.

gg. NASA-HDBK-4008, Programmable Logic Devices (PLD) Handbook, https://standards.nasa.gov/.

hh. NASA-HDBK-7009, NASA Handbook for Models and Simulations: An Implementation Guide for NASA-STD-7009, https://standards.nasa.gov/.

ii. NFS 1813.301-79, Supporting Federal Policies, Regulations, and NASA Procedural Requirements.

jj. NFS 1852.237-72, Access to Sensitive Information.

kk. NFS 1852.237-73 Release of Sensitive Information.

ll. NIST 800-146, Cloud Computing Synopsis and Recommendations.

mm. NIST 800-37, Risk Management Framework.

nn. NIST 800-53, Security and Privacy Controls for Federal Information Systems and Organizations.

oo. NPD 1200.1, NASA Internal Control.

pp. NPD 2540.1, Personal Use of Government Office Equipment Including Information Technology, https://nodis3.gsfc.nasa.gov/.

qq. NPD 2810.1, NASA Information Security Policy.

rr. NPR 2190.1, NASA Export Control Program, https://nodis3.gsfc.nasa.gov/.

ss. NPR 2210.1, Release of NASA Software, https://nodis3.gsfc.nasa.gov/.

tt. NPR 2830.1, NASA Enterprise Architecture Procedures, https://nodis3.gsfc.nasa.gov/.

uu. NPR 2841.1, Identity, Credential, and Access Management, https://nodis3.gsfc.nasa.gov/.

vv. NPR 7120.7, NASA Information Technology Program and Project Management Requirements.

ww. NPR 7120.10, Technical Standards for NASA Programs and Projects, https://nodis3.gsfc.nasa.gov/.

xx. NPR 7120.11, Health and Medical Technical Authority Implementation.

yy. NPR 7123.1, NASA Systems Engineering Processes and Requirements.

zz. NPR 8000.4, Agency Risk Management Procedural Requirements.

aaa. NPR 8735.2, Management of Government Quality Assurance Functions for NASA Contracts.

bbb. NPR 9250.1, Capital Asset Identification and Treatment, https://nodis3.gsfc.nasa.gov/.

ccc. NTRS ID 20160005787, “Quality Attributes for Mission Flight Software: A Reference for Architects.”

ddd. OMB Memo M-04-08, “Maximizing Use of SmartBuy and Avoiding Duplication Agency Activities with the President’s 24 E-Gov Initiatives.”

eee. OMB Memo M-04-16, “Software Acquisition.”

fff. OMB Memo M-15-14, “Management and Oversight of Federal Information Technology.”

ggg. OMB Memo M-16-12, “Category Management Policy 16-1: Improving the Acquisition and Management of Common Information Technology: Software Licensing.”