Comment:
Migration of unmigrated content due to installation of a new plugin
Tabsetup
0
1. The Requirement
1
2. Rationale
2
3. Guidance
3
4. Small Projects
4
5. Resources
5
6. Lessons Learned
6
7. Software Assurance
Div
id
tabs-1
1. Requirements
Excerpt
3.1.7 The project manager shall define and document the milestones at which the software developer(s) progress will be reviewed and audited.
1.1 Notes
NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.
1.2 History
Expand
title
Click here to view the history of this requirement: SWE-037 History
Include Page
SITE:SWE-037 History
SITE:SWE-037 History
1.3 Applicability Across Classes
Classes F and G are labeled as "X (not OTS)" which means that the project is required to meet this requirement for all software that is not considered off-the-shelf.
Applicable c
a
1
b
1
csc
1
c
1
d
1
dsc
1
e
0
f
1
g
0
h
0
Div
id
tabs-2
2. Rationale
For any software project, it is critical for management to review progress early and periodically to ensure the project remains on schedule, is progressing toward implementation of the requirements, and ultimately is addressing the customer's needs. It is also important for management to confirm periodically that the technical goals of the project are being achieved and that the technical direction of the project is appropriate (NPR 7123.1, NASA Systems Engineering Processes and Requirements).
Swerefn
refnum
041
Milestone reviews provide this type of management visibility into a project.
For software development that is acquired (supplied by a contractor), having regular progress reviews is even more important since these reviews are the keys to ensuring the contractor understood and will provide the product that NASA requested and that meets NASA's requirements for safety, quality, reliability, etc.
Milestone reviews can also serve to facilitate and ensure coordination between multiple development groups including development groups at multiple NASA Centers and contractors.
Div
id
tabs-3
3. Guidance
For acquired software development, milestone reviews are incorporated into the contract because the contract is the binding document for contractor performance and deliverables. Regardless of whether the development is in-house or contracted, the development agreement needs to contain, among other key elements, surveillance activities including monitoring activities, reviews, audits, decision points, meetings, known contract milestones, etc.
Other items related to milestone reviews to include in the development agreement are:
Review periods for deliverables.
The time for making corrections to resolve findings.
Formal reviews, such as those found in NPR 7123.1
Swerefn
refnum
041
, NPR 7120.5
Swerefn
refnum
082
, NPR 7120.7
Swerefn
refnum
264
(IT and Institutional Infrastructure), and NPR 7120.8
Swerefn
refnum
269
(Research and Technology).
Technical reviews.
Progress reviews.
Acceptance reviews.
Consult Center guidance on the following topics as Center guidance may provide insights into reviews and review topics to be considered for inclusion in the Statement of Work (SOW) and contract:
Acquisition.
Planning.
Scheduling.
Process Monitoring and Control (PMC).
See the 7.03 - Acquisition Guidance and 7.09 - Entrance and Exit Criteria topics in this Handbook for additional guidance on this topic. The references in 7.03 - Acquisition Guidance may provide additional guidance on project milestone reviews and topics for consideration. Topic 7.09 - Entrance and Exit Criteria describes inputs, material that will be reviewed, and outputs for each life-cycle milestone review which may be useful as input to the development of checklists for these reviews. Consult the NPR 7120 family of requirements documents for definitions of milestones and approaches for different project types.
Additional information and resources regarding software milestones are available in Software Processes Across NASA (SPAN), accessible to NASA users from the SPAN tab in this Handbook.
Keep in mind that reviews not included in the contract may be difficult to require of the contractor, so it is important to ensure the SOW and other contract elements are reviewed by the proper project management and/or technical authority for completeness.
Div
id
tabs-4
4. Small Projects
Project review plans and milestones are covered in NPR 7120.5
Swerefn
refnum
082
, NPR 7120.7
Swerefn
refnum
264
, NPR 7120.8
Swerefn
refnum
269
, and NPR 7123.1
Swerefn
refnum
041
. Projects are to ensure that software components are a part of specific project milestone reviews. Small projects need to determine a review process that meets the project requirements and contains adequate content to provide the greatest insights into progress toward the project's technical goals and the technical direction of the project.
Div
id
tabs-5
5. Resources
5.1 References
refstable
Show If
group
confluence-users
Panel
titleColor
red
title
Visible to editors only
Enter the necessary modifications to be made in the table below:
SWEREFs to be added
SWEREFS to be deleted
SWEREFs called out in text: 041, 082, 264, 269, 528
SWEREFs NOT called out in text but listed as germane: 048
5.2 Tools
Include Page
Tools Table Statement
Tools Table Statement
Div
id
tabs-6
6. Lessons Learned
6.1 NASA Lessons Learned
A documented lesson from the NASA Lessons Learned database notes the following:
Acquisition and Oversight of Contracted Software Development (1999). Lesson Number 0921
Swerefn
refnum
528
: Tailorable acquisition management and oversight processes for NASA contracted software development are essential to ensure that customers receive a quality product. A documented lesson from the NASA Lessons Learned database includes a cause of the loss of a mission "the lack of a controlled and effective process for acquisition of contractor-developed, mission-critical software." In this particular case, the quality of the contractor's product was not monitored as it would have been if the proper milestones for reviewing and auditing contractor progress were in place.
6.2 Other Lessons Learned
No other Lessons Learned have currently been identified for this requirement.
Div
id
tabs-7
7. Software Assurance
Excerpt Include
SWE-037 - Software Milestones
SWE-037 - Software Milestones
7.1 Tasking for Software Assurance
Confirm that milestones for reviewing and auditing software developer progress are defined and documented.
Participate in project milestones reviews.
7.2 Software Assurance Products
Software Assurance Status Reports
SA audit schedule consistent with the project schedule.
List of any issues//risks identified with schedule or developer progress.
Note
title
Objective Evidence
Milestone review plans.
Definition for software products lifecycle entrance and exit criteria.
Evidence of SA participation in milestone reviews.
Expand
title
Definition of objective evidence
Include Page
SITE:Definition of Objective Evidence
SITE:Definition of Objective Evidence
SA Products for Milestone Reviews
Review
Software Assurance Product for Review
All Reviews
SA's general assessment of the status and quality of the software and safety activities
Any schedule/progress, quality, or safety concerns
High-level results of any audits, peer reviews, assessments, or analyses
Additional Software Assurance /Safety Products for Milestone Reviews
System Requirements Review (SRR)
Independent SA Software Classification or Concurrence on Software Engineering's Software Classification
SA Safety Criticality Determination
Preliminary Hazard Analysis
Preliminary SA Plan, SA schedule, and SA processes
Software Requirements Review (SwRR)
Software Assurance/Quality Plan
Preliminary Safety Plan
SA Requirements Assessment, including any issues with bidirectional traceability
Mission Definition Review (MDR)
Updated Software Assurance/Safety Plan
Preliminary SA Safety Analysis
SA status, including results from any assessments, audits, peer reviews (See all reviews above)
System Definition Review (SDR)
Software Safety Analysis
Software Assurance/Safety Plan (for baselining)
SA status, including results from any assessments, audits, peer reviews (See all reviews above)
Preliminary Design Review (PDR)
Software Safety Analysis, including FTA, FMEA (if done)
SA Compliance Matrix
SA status, including results from any assessments, audits, peer reviews (See all reviews above)
Critical Design Review (CDR)
Hazard Analysis
SA Safety Plan with verifications
SA assessment of design
SA analysis of software progress, metrics
SA status of safety and assurance activities, including results from any assessments, audits, peer reviews
Production Readiness Review (PRR)
SA status (See all reviews above)
System Integration Review (SIR)
SA status (as in all reviews), including any concerns with safety or the integration process
SA analysis of static code analysis results
SA assessment of integration plans, procedures, or handling of safety requirements
Status of system discrepancies, system test results
Test Readiness Review (TRR)
SA assessment of test readiness, test procedures, witnessing plans (or SA participation plans)
SA analysis of software metrics, software progress
SA results of any code assessments, audits, peer reviews
SA results of Version Description Document (VDD) and Functional Configuration Audits (FCA)
System Acceptance Review (SAR)
SA Status
Results of configuration audits: Functional Configuration Audits( FCAs) and Physical Configuration Audits (PCAs)
Status of documentation, previous testing, including assessment of metrics on non-conformance/Problem Reports (PRs)/Discrepancy Reports (DRs)
Operational Readiness Review (ORR)
Physical Configuration Audit (PCA) results
SA assessment of V&V status and documentation
SA status on safety and security activities
SA assessment of readiness for operations and maintenance
Results of any other audits, assessments, and metrics analysis
Flight Readiness Review (FRR)
SA assessment of readiness for flight
Any SA concerns with safety or security
SA sign-offs on certification packages
7.3 Metrics
# of Non-Conformances from reviews (Open vs. Closed; # of days Open)
7.4 Guidance
List of tasks for the software development required for the project’s software developers. - look at the software development schedule milestones (see 7.08 - Maturity of Life-Cycle Products at Milestone Reviews, software products required, and software processes used. Related to the SA tasks on SWE-036.
