See edit history of this section
Post feedback on this section
- 1. The Requirement
- 2. Rationale
- 3. Guidance
- 4. Small Projects
- 5. Resources
- 6. Lessons Learned
- 7. Software Assurance
1. Requirements
5.1.8 The project manager shall establish and implement procedures for the storage, handling, delivery, release, and maintenance of deliverable software products.
1.1 Notes
NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.
1.2 History
1.3 Applicability Across Classes
Class A B C D E F Applicable?
Key: - Applicable | - Not Applicable
2. Rationale
Given that software is considered a significant Agency investment, it is important to ensure that the delivered software is created and maintained from a controlled repository. Configuration management (CM) processes and controls provide the rigor and organization necessary for developers and their customers to have confidence that all changes to the code and documents are included in the released products. It is also important that the released product is stored, maintained, and delivered following a repeatable, controlled process.
3. Guidance
3.1 Definitions
- particular version of a configuration item that is made available for a specific purpose
- collection of new or changed configuration items that are tested and introduced into a live environment together
- (IEEE Std 828-2012, IEEE Standard for Configuration Management in Systems and Software Engineering 216, 2.1)
- collection of one or more new or changed configuration items deployed into the live environment as a result of one or more changes
- (ISO/IEC 19770-5:2015(en) Information technology — IT asset management — Overview and vocabulary 059, 3.28)
- software version that is made formally available to a wider community
- (IEEE Std 828-2012, IEEE Standard for Configuration Management in Systems and Software Engineering 216, 2.1)
- delivered version of an application which includes all or part of an application
- (IEEE Std 828-2012, IEEE Standard for Configuration Management in Systems and Software Engineering 216, 2.1)
- set of grouped change requests, established in the Application Change Management Process, which are designed, developed, tested, and deployed as a cohesive whole
- (ISO/IEC 16350:2015 - Information technology — Systems and software engineering — Application management 412, 4.28)
- distribution of a product increment to a customer or users
Example: source code, code for execution, or multiple software assets packaged into an internal production release and tested for a target platform, test release
Release management includes defining acceptable quality levels for release, authority to authorize the release, and release procedures. See Also: version
- (1) initial release or re-release of a computer software configuration item, associated with a complete compilation or recompilation of the computer software configuration item (IEEE Std 828-2012, IEEE Standard for Configuration Management in Systems and Software Engineering 216, 2.1)
- (2) initial release or complete re-release of a document, as opposed to a revision resulting from issuing change pages to a previous release (IEEE Std 828-2012, IEEE Standard for Configuration Management in Systems and Software Engineering 216, 2.1)
- (3) operational software product that differs from similar products in terms of capability, environmental requirements, and configuration (ISO/IEC/IEEE 24765:2017 Systems and software engineering – Vocabulary 230)
- (4) identified instance of a configuration item (ISO/IEC TR 18018:2010 - Information technology — Systems and software engineering — Guide for configuration management tool capabilities 429, 3.15)
- (5) unique string of number and letter values indicating a unique revision of an item (ISO/IEC 19770-5:2015(en) Information technology — IT asset management — Overview and vocabulary 059, 3.54)
Versions often identify revisions of software that provide unique functionality or fixes. A version typically has multiple parts, such as a major version, indicating large changes in functionality or user interface changes, and a minor version, indicating smaller changes in functionality or user interface changes.
3.2 Release Management Procedures
As with other CM activities, the CM plan includes the plans and references the procedures for software release management.
When developing procedures for release management, address all of the following:
- Preparation of the release package.
- Creation and delivery of the release package.
- Storage and maintenance of the release package.
Release management procedures may vary depending on the recipient of the release. Internal releases, such as baselines released for testing, will most likely not require the same set of release activities and considerations as formal releases to external customers.
See also SWE-077 - Deliver Software Products, SWE-194 - Delivery Requirements Verification,
3.2.1 Preparation of the Release Package
The SMA (Safety and Mission Assurance) Technical Excellence Program (STEP) Level 2 Software Configuration Management and Data Management course taught by the Westfall Team 343 provides a good list of release planning and scheduling activities.
A checklist of activities to complete may be useful as part of the preparation to create the release package. A list of activities to consider includes:
- Ensure the proper approvals have been documented and received, including:
- Software assurance - "Software assurance shall provide objective evidence to the project and NASA Safety and Mission Assurance (SMA) of the software's readiness for operational release." 278
- Release authority – Change Control Board (CCB) or other authorized "owner."
- Ensure any required acceptance data package has been prepared (see NASA-GB-8719.13, NASA Software Safety Guidebook, 276 for typical content information), including the Version Description Document (5.16 - VDD - Version Description Document).
- Ensure all required configuration audits have been completed (SWE-084 - Configuration Audits).
- Ensure all approved deviations and waivers are documented.
- Ensure all change requests have been completed and verified.
- Ensure all documents and training materials are complete, including installation and any special installation needs/support, customization and configuration documents, user and operator guides, and release notes.
- Ensure all legal issues, such as licensing or export regulations (e.g., International Traffic in Arms Regulations (ITAR)), are addressed, as applicable.
- Ensure any "ready to ship" reviews are completed.
- Ensure all installation sites are prepared to receive and install the release, conducting any pre-installation visits as appropriate.
- Ensure required support personnel is trained and ready to address issues related to the installed release.
See also SWE-083 - Status Accounting, SWE-081 - Identify Software CM Items. See also Topic 5.01 - CR-PR - Software Change Request - Problem Report, 5.06 - SCMP - Software Configuration Management Plan,
3.2.2 Creation and Delivery of the Release Package
Procedures for creating the release package are used once the preparation steps have been completed and it is time to create the release package. Typically, there is a master copy of the release package, and copies are distributed to customers. Depending on Center policy, the master may be created by the CM group and the copies created, packaged, and shipped by another group. Whatever process is used needs to be clearly defined in the release management procedures.
When developing those procedures, consider the following:
- Identify the scope of the release, including the full set of configuration items (CIs) that are to be included, their versions, and revisions.
- Identify the tools to be used to create the release, including compilers and linkers.
- Identify the software to be used to create the release, including the operating system, macros, and libraries.
- Identify software and tool options to be used (compiler options, environmental parameters).
- Identify the procedures for creating the master copy of the release or reference them if captured elsewhere.
- Identify who generates the master copy of the release package.
- Document the format, layout, and media for the master.
- Document the verification process to confirm the master contains the proper CI's.
- Identify the media to be used for the delivery of copies.
- Document replication procedures to be used to generate copies of the master.
- Document verification procedures to be used to confirm the copies match the master (keep in mind that compilers can insert dates and times, so byte-by-byte compares need to take this into account).
- Document any virus checks that need to be run on the copies before delivery to the customer.
- Document any testing to be performed at the customer site (e.g., regression testing).
When developing procedures for delivery of the created package, consider the following:
- Document whether the release is full, the partial release which requires a previous full release to be installed first, or a patch; if all types will be used, procedures for creating and installing each need to be created.
- Document delivery methods and procedures, including shipping methods, if required.
- Document security measures to be used when handling and shipping the release.
- Determine an installation schedule that works with the customer's schedule.
- Document responsibilities for performing installation and installation testing.
- Document responsibilities for configuring and/or customizing the installed software.
- Document plans to revert to an earlier release of the software, as applicable.
3.2.3 Storage and Maintenance of the Release Package
As part of release management, the master needs to be safely and securely stored following documented procedures. When developing procedures for storing and maintaining the release package, consider the following:
- Document the retention period; e.g., "master copies of all configuration items in a release and the release itself shall be maintained for the life of the product" (IEEE Std 828-2012, IEEE Standard for Configuration Management in Systems and Software Engineering 216)
- Document how to place the master into the CM system with its unique identifier.
- Document access restrictions.
- Document or reference any specific procedures for storing code and documentation for safety or security-critical functions.
- Identify release records, such as the Version Description Document (VDD), to be captured and stored with the release, as applicable.
When defining activities for release management, consider coordinating or applying the same concepts as part of data management activities. A basic description of data management is provided in SWE-079 - Develop CM Plan.
3.3 Additional Guidance
Additional guidance related to this requirement may be found in the following materials in this Handbook:
3.4 Center Process Asset Libraries
SPAN - Software Processes Across NASA
SPAN contains links to Center managed Process Asset Libraries. Consult these Process Asset Libraries (PALs) for Center-specific guidance including processes, forms, checklists, training, and templates related to Software Development. See SPAN in the Software Engineering Community of NEN. Available to NASA only. https://nen.nasa.gov/web/software/wiki 197
See the following link(s) in SPAN for process assets from contributing Centers (NASA Only).
SPAN Links |
---|
4. Small Projects
Projects with limited budgets may choose to follow a common Center or project-level release management procedure rather than have separate procedures for each project. Slight modifications may be required for each project, but the overall master process would not have to be developed or maintained on a per-project basis.
5. Resources
5.1 References
- (SWEREF-001) Software Development Process Description Document, EI32-OI-001, Revision R, Flight and Ground Software Division, Marshall Space Flight Center (MSFC), 2010. See Chapter 13. This NASA-specific information and resource is available in Software Processes Across NASA (SPAN), accessible to NASA users from the SPAN tab in this Handbook.
- (SWEREF-059) ISO/IEC 19770-5:2015(en), Information technology — IT asset management —
- (SWEREF-083) NPR 7150.2D, Effective Date: March 08, 2022, Expiration Date: March 08, 2027 https://nodis3.gsfc.nasa.gov/displayDir.cfm?t=NPR&c=7150&s=2D Contains link to full text copy in PDF format. Search for "SWEREF-083" for links to old NPR7150.2 copies.
- (SWEREF-197) Software Processes Across NASA (SPAN) web site in NEN SPAN is a compendium of Processes, Procedures, Job Aids, Examples and other recommended best practices.
- (SWEREF-212) IEEE Computer Society, IEEE STD 1042-1987, 1987. This link requires an account on the NASA START (AGCY NTSS) system (https://standards.nasa.gov ). Once logged in, users can access Standards Organizations, IEEE and then search to get to authorized copies of IEEE standards.
- (SWEREF-216) IEEE STD IEEE 828-2012, 2012., NASA users can access IEEE standards via the NASA Technical Standards System located at https://standards.nasa.gov/. Once logged in, search to get to authorized copies of IEEE standards.
- (SWEREF-230) ISO/IEC/IEEE 24765:2017 It was prepared to collect and standardize terminology. Copy attached.
- (SWEREF-271) NASA STD 8719.13 (Rev C ) , Document Date: 2013-05-07
- (SWEREF-276) NASA-GB-8719.13, NASA, 2004. Access NASA-GB-8719.13 directly: https://swehb.nasa.gov/download/attachments/16450020/nasa-gb-871913.pdf?api=v2
- (SWEREF-278) NASA-STD-8739.8B , NASA TECHNICAL STANDARD, Approved 2022-09-08 Superseding "NASA-STD-8739.8A,
- (SWEREF-337) Souppaya, Murugiah, Scarfone, Karen, NIST Special Publication NIST SP 800-40r4, April, 2022
- (SWEREF-343) This NASA-specific information and resource is available in at the System for Administration, Training, and Educational Resources for NASA (SATERN), accessible to NASA-users at https://saterninfo.nasa.gov/.
- (SWEREF-373) NPR 2210.1C, Space Technology Mission Directorate, Effective Date: August 11, 2010, Expiration Date: January 11, 2022
- (SWEREF-412) ISO 16350:2015 establishes a common framework for application management processes with well-defined terminology that can be referenced by the software industry.
- (SWEREF-415) ISO/IEC/IEEE 12207:2017 also provides processes that can be employed for defining, controlling, and improving software life cycle processes within an organization or a project.
- (SWEREF-423) This document provides an overview of agile readiness factors that are likely to determine whether an organization, project, product or team is ready to start the transition to using an agile approach to their system and software development and maintenance activities.
- (SWEREF-429) ISO/IEC TR 18018:2010 provides guidance in the evaluation and selection for CM tools during acquisition.
- (SWEREF-526) Public Lessons Learned Entry: 838.
- (SWEREF-541) Public Lessons Learned Entry: 1130.
- (SWEREF-574) Public Lessons Learned Entry: 2476.
- (SWEREF-580) COTS Change Processing Lessons Learned Entry:3457. No longer available
- (SWEREF-585) In NASA Engineering Network.
- (SWEREF-586) Public Lessons Learned Entry: 559.
- (SWEREF-587) Public Lessons Learned Entry: 2158.
- (SWEREF-588) Public Lessons Learned Entry: 4516.
5.2 Tools
NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN.
The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool. The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.
6. Lessons Learned
6.1 NASA Lessons Learned
The NASA Lesson Learned database contains the following lesson learned related to release management:
- Computer Hardware-Software/International Space Station/Software Configuration Management. Lesson No: 1130 541: Although it was "grandfathered" out of NPR 7150.2 compliance, the experience regarding source code for the International Space Station (ISS) is an important caveat when establishing software supplier agreements. NASA does not have source code access for all partners' deliveries for the ISS. The partners cite their concerns that the delivery of source code could compromise their contractors' proprietary data. The ISS has initiated discussions with all partners to reach an agreement on what level of source code visibility is necessary to ensure adequate knowledge by the control centers for on-orbit anomaly resolution. It is not clear how much extra effort these discussions have taken.
- Place Flight Scripts Under Configuration Management Prior to ORT (Project attention to configuration control). Lesson Number 2476 574: "When flight scripts developed independently by project personnel are not placed under configuration control early enough in-flight software development, multiple versions of the scripts tend to proliferate and cause confusion and delays. The MER, Juno, and GRAIL projects demonstrated the utility of placing flight scripts under change control prior to ORT."
- COTS Change Processing. Lesson Number 3457 580: “The time-sensitive nature of commercial-off-the-shelf (COTS) configuration changes such as the application of routine security patches and vendor updates/upgrades could not be effectively managed using the time-consuming configuration management practices typical of a custom environment. Systems that use both COTS and custom hardware and software products must adopt configuration management practices that effectively address the unique requirements of both product categories."
- Software Design for Maintainability. Lesson Number 0838 526: Impact of Non-Practice: "Because of increases in the size and complexity of software products, software maintenance tasks have become increasingly more difficult. Software maintenance should not be a design afterthought; it should be possible for software maintainers to enhance the product without tearing down and rebuilding the majority of code."
- International Space Station (ISS)/Multiple Element Integrated Testing (MEIT)/Software Configuration Management. Lesson Number 1165 585 : “Due to the rapid pace of ISS assembly launches and the many and varied resulting configurations, MultiElement Integration Testing MEIT with operational loads of Portable Computer System PCS software is limited and, in some cases, may only be accomplished in the brief time allocated for regression testing.”
- Redundant Verification of Critical Command Timing (1995). Lesson Number 559 586: “When a new mission software release was uploaded to the spacecraft, the inflight upload failed to include a software patch that had been written to fix a defective countdown timer. Because an independent watchdog timer was planned but never implemented due to constrained project resources, the thrusters continued to fire after the desired shutdown time and the mission was terminated. Recommendations centered on the need for rigorous software configuration management, a watchdog timer to terminate operations, and testbed verification of in-flight software updates.”
- Reuse of Analysis Software. Lesson Number 2158 587: “When a project plans to reuse existing metrology software for a new application, a thorough independent review of the software and its new interfaces should be conducted.”
- Institutional Configuration Management Organization. Lesson Number 4516 588: “Configuration and Data Management CDM is performed at the Kennedy Space Center KSC for hardware and software by the different programs and projects independently of the Center/Institution. What is important for CDM is having consistency in policy, guidelines, processes, and tools to provide seamless and efficient transaction of configuration information. A single institutional CDM organization can provide consistent guidance and direction to comply with agency, program, and project CDM requirements.”
6.2 Other Lessons Learned
No other Lessons Learned have currently been identified for this requirement.
7. Software Assurance
7.1 Tasking for Software Assurance
1. Confirm that the project establishes procedures for storage, processing, distribution, release, and support of deliverable software products.
7.2 Software Assurance Products
- Software Configuration Management Baseline and Process/Procedure Audit Report
- Results and findings from audits on CM process performance, including any risks or issues.
Objective Evidence
- Software Problem reporting or defect tracking data
- Software configuration management system data
- Software assurance audit results of the configuration management processes.
- Software data file location data
- Software version description document(s).
7.3 Metrics
- # of Compliance Audits planned vs. # of Compliance Audits performed.
- # of software process Non-Conformances by life cycle phase over time.
- # of Non-Conformances identified in release documentation (Open, Closed).
- # of process Non-Conformances (e.g., activities not performed) identified by SA vs. # accepted by the project.
- Trends of # Open vs. # Closed over time.
- # of Non-Conformances per audit (including findings from the process and compliance audits, and process maturity).
- # of Open vs. Closed Audit Non-Conformances over time,
- Trends of # of Non-Conformances from audits over time (Include counts from process and standards audits and work product audits.).
See also Topic 8.18 - SA Suggested Metrics.
7.4 Guidance
For Task 1, software assurance will review project documentation and confirm that procedures are established for storing, handling, release, delivery, and maintenance. These processes and procedures may be found in some of the following documents including the software development/management plan, data management plans, configuration management plan, and maintenance plans or they may reference other project documents or Center document describing their processes. If Center procedures are referenced, they should be tailored for the project-specific details. Any project-established procedures should be checked for compliance with the requirements NPR 7150.2 083 and any Center requirements.
For Task 2, perform various types of audits to confirm that the procedures reviewed in Task 1 are being followed. These audits would include process/procedure audits to ensure that the activities specified in the procedures are being done. Other audits should include audits done before delivery such as the Functional Configuration Audit (FCA) and Physical Configuration Audit (PCA). The process for configuration management should be audited periodically to ensure the project data is being handled and stored properly. For more information on planning and executing audits, see 8.12 - Basics of Software Auditing. For more information FCAs and PCAs. see SWE-084 - Configuration Audits.
In addition to these two tasks, for releases and delivery, software assurance will sign off on the delivery documentation such as the FCA and PCA audits to indicate that all required activities have been completed and the required conditions have been met. Some of the activities and conditions that need to be considered are below:
- The software is accepted. Generally, the software is considered when it has met the acceptance criteria as defined in the test plan, satisfies the technical requirements, and meets the requirements in NPR 7150.2, NASA-STD-8739.8 278, and any Center-level requirements, and any additional contract, MOA, or MOU software requirements. Software acceptance can be rolled into the system acceptance as long as the software has all been tested and signed off by software assurance as concurrence indicates the software meets the requirements for SW quality, SW safety, SW reliability, SW security, and SW V&V, and SW IV&V.
- Software assurance has evaluated and concurred that Project/Provider has satisfactorily completed SW and system verification and validation plans and procedures and that test results are acceptable for the target environment and the intended software use.
- Software assurance has verified SW deliveries by conducting or participating in (and signing off on) the execution of configuration audits, such as Functional Configuration Audit (FCA) and Physical Configuration Audit (PCA). Software Assurance accepts or rejects SW deliveries based on the state of the SW resolution of DRs/PRs and any outstanding risks and satisfactory completion of any safety and reliability verifications.
- Software assurance has verified the integrity of SW installations against the delivery documentation, data checks, operational set-up, and configuration managed versions.
- Software assurance has assured that the operations and maintenance environment is prepared for software delivery, installation, operations, and maintenance including the transfer or acquisition of resources (e.g., licenses, test benches, simulators) necessary to conduct operations and maintenance.
SA should assure that the delivered SW is placed under configuration control and for OTS or reuse SW, that all processes for baselining and assuring the acquired SW, including any wrappers, glueware, or applications that may run on the OTS, are secure, documented and under configuration control.
- Software assurance confirms that any operations or maintenance documents are complete, usable, and delivered, including such information as licenses, simulators, models, COTS, open-source, tools, or reuse code incorporated or used, as applicable.
Software assurance will do the following activities and deliverables for the operations, maintenance, and retirement of the software:
- Assure that the processes for maintenance and retirement are being followed as per their plans
- Assure that the plans for maintenance, operations, and retirement are complete.
- Review changes approved for implementation during maintenance and confirm that:
- Changes will not impact Project risk posture concerning quality, safety, reliability, security, or NPR 7150.2 compliance
- Changes are being tested thoroughly, including regression testing
- Approved changes are documented and reflected in “as-built” documentation
- Document and raise to the Project any issues, risks, or concerns
- Confirm that retirement is carried out as per the retirement plan and that:
- All software and documentation are archived
- Licenses are transferred or canceled
- Any equipment or furniture in associated labs is properly transferred or disposed of
- Report on any issues, concerns
Every task that involves performing an audit should also clarify that all audit findings are promptly shared with the project and will be addressed in the handbook guidance.
7.5 Additional Guidance
Additional guidance related to this requirement may be found in the following materials in this Handbook: