1. Requirements

1.1 Notes

This includes operating systems, run-time systems, device drivers, code generators, compilers, math libraries, and build and Configuration Management (CM) tools. It should be performed pre-flight, with mandatory code audits for critical defects.

1.2 History

1.3 Applicability Across Classes

2. Rationale

Software components that are used to build the software product (e.g., compilers) or become a part of the software (e.g., Operating Systems) can introduce unexpected defects in the delivered product.  Whenever non-conformances in these products are discovered, a thorough assessment is required to identify any other potential impacts.    

3. Guidance

Non-conformances discovered in COTS, GOTS, MOTS, OSS, and reused software components are particularly difficult to diagnose.  Determining the risk to the overall system, due to errors in these acquired software components is challenging.  Most COTS, OSS, and some GOTS, MOTS, and reused software components share lists of known defects and non-conformances.  The intent of the requirement is for the user of the software to verify if a site or list of non-conformances or reported bugs is maintained by the developing organization and to review the list of known non-conformances or report bugs to see if the non-conformances or reported bugs could or do impact the software component.  Most commercial products and open-source software have a site that shows a list of well know bugs or non-conformances. The requirement is to research the information and see if any of the known bugs impact the software component being used by the project. Thorough assessments, including qualified software and systems engineers, of all reported non-conformances in these products, are essential for Project Managers.  Additionally, non-conformances in these products should be reported back to the supplier for analysis and potential corrections in future versions.

4. Small Projects

No additional guidance is available for small projects.

5. Resources

5.1 References

5.2 Tools

6. Lessons Learned

6.1 NASA Lessons Learned

No Lessons Learned have currently been identified for this requirement.

6.2 Other Lessons Learned

No other Lessons Learned have currently been identified for this requirement.

7. Software Assurance

7.1 Tasking for Software Assurance

1. Confirm the evaluations of reported non-conformances for all COTS, GOTS, MOTS, OSS, or reused software components are occurring throughout the project life-cycle.

2. Assess the impact of non-conformances on the safety, quality, and reliability of the project software.

7.2 Software Assurance Products

• Software Design Analysis
• Source Code Analysis
• Verification Activities Analysis
• SA impact assessment of non-conformances on software quality (safety, quality, reliability.)

  
  

  Note
  title Objective Evidence
  Software defect or problem reporting data for all COTS, GOTS, MOTS, OSS, or reused software components Software configuration management data Software assurance audit results on the change management or defect management processes Software milestone results Software version description documents Software control board data or presentations

  Expand
  title Definition of objective evidence
  Include Page SITE:Definition of Objective Evidence SITE:Definition of Objective Evidence

7.3 Metrics

• Total # of Non-Conformances over time (Open, Closed, # of days Open, and Severity of Open)
• # of Non-Conformances in the current reporting period (Open, Closed, Severity)
• # of Non-Conformances identified in source code products used (Open, Closed)
• # of safety-related Non-Conformances
• # of Non-Conformances identified in embedded COTS, GOT, MOTS, OSS, or reused components in-ground or flight software vs. # of Non-Conformances successfully closed

7.4 Guidance

Software Assurance will verify that the project is receiving the report of non-conformances for all of the COTS, GOTS, MOTS, OSS, or reused software that is being used by the project. Check that these reported non-conformances reports are being received periodically and that the project is reviewing them to see if there are any impacts to their project. If fixes to any of the COTS, GOTS, MOTS, OSS, or reused software that might impact the project, the project should be implementing these changes. Verify that any necessary changes are recorded in the project discrepancy database and track these entries to closure.

Review the lists of COTS, GOTS, MOTS, OSS, or reused software non-conformances and determine whether there might be any impacts on the software’s safety, quality, or reliability. If the non-conformance is in an area of safety-critical software, review the associated hazard analysis to determine the impact of the non-conformance. In non-safety critical areas of the code, assess what impact the non-conformance would have—Would it cause an incorrect value to be computed? Or prevent critical functionality from working properly? Or severely impact performance? Or is it something like spelling on a display? Reviewing the functionality of the code in the area of the non-conformance, thinking through the operational scenarios, or looking at redundancy for the code area can help in determining the impacts of the non-conformance. If the source code is available, it may also be helpful to run code analyzers to see if the known problems are discovered and how the analyzer classifies the non-conformances. If there are non-trivial impacts, verify that these non-conformances have been added to the project discrepancy database and track them to closure.