bannerd


SWE-054 - Corrective Action for Inconsistencies

1. Requirements

4.1.6 The project manager shall identify, initiate corrective actions, and track until closure inconsistencies among requirements, project plans, and software products. 

1.1 Notes

NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.

1.2 History

SWE-054 - Last used in rev NPR 7150.2D

RevSWE Statement
A

3.1.2.2 The project shall identify, initiate corrective actions, and track until closure inconsistencies among requirements, project plans, and software products.

Difference between A and B

No change

B

4.1.3.2 The project manager shall identify, initiate corrective actions, and track until closure inconsistencies among requirements, project plans, and software products.

Difference between B and C

No change

C

4.1.6 The project manager shall identify, initiate corrective actions, and track until closure inconsistencies among requirements, project plans, and software products. 

Difference between C and DNo change
D

4.1.6 The project manager shall identify, initiate corrective actions, and track until closure inconsistencies among requirements, project plans, and software products. 



1.3 Applicability Across Classes

If Class D software is safety-critical, this requirement applies to the safety-critical aspects of the software.

Class

     A      

     B      

     C      

     D      

     E      

     F      

Applicable?

   

   

   

   

   

   

Key:    - Applicable | - Not Applicable


2. Rationale

Requirements are the basis for a project. They identify the need to be addressed, the behavior of the system, and the constraints under which the problem is to be solved. They also specify the product to be delivered by the software provider. Software project plans describe how the project will create a software product that fulfills those requirements. Any inconsistencies among these three - requirements, project plans, and software products – will most likely result in a product that does not meet its objectives. These inconsistencies are identified and rectified via corrective actions. Corrective actions address not only new or changed requirements, but also failures and defects in the work products (requirements, project plans, and software products). Corrective actions need to be analyzed to determine the impact that the change will have on the work product, related work products, budget, and schedule. Corrective actions need to be tracked until closure to ensure decisions regarding the closure of those actions are followed through and to prevent the problems described in those corrective actions from propagating through the project or recurring.

Identifying and correcting inconsistencies early and continuously means that a project is more likely to produce a result that satisfies the need for which it was established. Identifying and correcting inconsistencies early (rather than later) also reduces overall project cost as inconsistencies will not propagate forward in the project life cycle (which would require rework).

3. Guidance

Typically, the corrective action process is described in a plan, such as the software development or software management plan (5.08 - SDP-SMP - Software Development - Management Plan) or the Software Assurance Plan (8.51 - Software Assurance Plan). Follow this documented process to capture and track to closure inconsistencies among requirements, plans, and software products.

A recommended practice for this requirement is that all corrective actions be formally submitted with descriptions of the desired modifications to work products. If corrective actions are not documented consistently, they are difficult to analyze and track. A database provides a flexible environment for storing and tracking corrective actions.

See also SWE-024 - Plan Tracking

3.1 Identify inconsistencies among requirements, project plans, and software products

Suggested activities to identify inconsistencies and ensure that plans and activities or work products remain consistent with requirements, include:

  • Review plans, activities, and work products for consistency with requirements and changes made to them.
  • Record inconsistencies and their source. 
  • Initiate and record any necessary corrective actions and communicate results to affected stakeholders. 
  • Maintaining bidirectional traceability is critical to maintaining consistency between requirements, work products, and plans. 
  • Identify any changes that should be made to plans and projects resulting from changes to the requirements. 

When looking for inconsistencies, consider these "warning signs" or potential causes:

  • Unstable requirements.
  • Unclear, incomplete, inconsistent, non-cohesive requirements.
  • Incomplete project plans or plans developed before requirements stabilize.
  • Budgetary issues that result in changes to staff and/or the requirements.
  • Inadequate communications among the development team, management, customers, contractors, and other stakeholders.
  • Inadequate change or configuration management procedures.
  • Inexperienced staff.
  • Personnel turnover.

Inconsistencies among plans, requirements, and the resulting software products need to be identified throughout the project life cycle.  One way to ensure this activity is not forgotten is to conduct periodic reviews of requirements to ensure the requirements, project plans, and software products are consistent. See also SWE-053 - Manage Requirements Changes and SWE-080 - Track and Evaluate ChangesSWE-050 - Software Requirements


At a minimum, project teams need to note which requirements change and where those requirements flow into plans and products so those plans and products can be reconciled with the changed requirements. This needs to be a standard part of the change control process when a change is being evaluated. 

Before corrective action can be taken, consider performing an analysis to identify and weigh options when the corrective action is not readily apparent. This analysis could be similar to that used for change requests and problem reports (see 5.01 - CR-PR - Software Change Request - Problem Report).

Test plans/procedures need to be modified to reflect requirement changes and resulting implementation changes.  Testing of code changes should be conducted, including regression testing.  If the software is safety-critical, a full set of regression tests should be run to ensure that there was no impact on the safety-critical functions.  Refer to Software Assurance section.

3.2 Initiate corrective actions and track until closure

Sample corrective actions include:

  • Split development into multiple releases, addressing unstable requirements later.
  • Use more experienced or senior personnel.
  • Stabilize project personnel, i.e., reduce turnover.
  • Audit project processes and act on the findings.


NASA/SP-2016-6105 Rev2, NASA Systems Engineering Handbook

5.4.1.2.3 Analyze Product Validation Results - Validation Deficiencies
"Care should be exercised to ensure that the corrective actions identified to remove validation deficiencies do not conflict with the baselined stakeholder expectations without first coordinating such changes with the appropriate stakeholders."
273

To initiate corrective actions and track them to closure, consider the following for implementation on the project:

  • A system or process for entering and tracking corrective actions (e.g., Problem Report and Corrective Action (PRACA), change request/problem reporting system or tool).
  • A corrective action review process - typically involves a review panel or board including engineers and assurance personnel (e.g., Configuration Control Board (CCB)).
  • A corrective action implementation process, including updates to test procedures and identification of regression tests.
  • A time frame for resolution (i.e., number of days or weeks).
  • An escalation procedure (e.g., report to Project Manager (PM)).
  • A procedure to archive actions and conclusions for reference, and input to future projects.

3.3 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook:

3.4 Center Process Asset Libraries

SPAN - Software Processes Across NASA
SPAN contains links to Center managed Process Asset Libraries. Consult these Process Asset Libraries (PALs) for Center-specific guidance including processes, forms, checklists, training, and templates related to Software Development. See SPAN in the Software Engineering Community of NEN. Available to NASA only. https://nen.nasa.gov/web/software/wiki  197

See the following link(s) in SPAN for process assets from contributing Centers (NASA Only). 

SPAN Links

4. Small Projects

No additional guidance is available for small projects.

5. Resources

5.1 References

5.2 Tools


Tools to aid in compliance with this SWE, if any, may be found in the Tools Library in the NASA Engineering Network (NEN). 

NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN. 

The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool.  The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.

6. Lessons Learned

6.1 NASA Lessons Learned

The NASA Lessons Learned database contains the following lessons learned related to ensuring plans and products remain aligned with requirements:

  • Risk Assessment in Software Development Projects (Uncertainty caused by changing requirements.) Lesson Number 1321 549: "Even with expert and experienced programmers, each new software program presents new technical challenges. Changing methodology and requirements during the design phase of a software project adds uncertainty to the project."
  • Deficiencies in Mission Critical Software Development for Mars Climate Orbiter (1999) (Inconsistency led to the loss of mission.) Lesson Number 0740 521: "The root cause of the MCO mission loss was an error in the "Sm_forces" program output files, which were delivered to the navigation team in English units (pounds-force seconds) instead of the specified metric units (Newton-seconds). Comply with preferred software review practices, identify mission-critical software (for which staff must participate in major design reviews, walkthroughs, and review of acceptance test results), train personnel in software walkthroughs, and verify consistent engineering units on all parameters.

6.2 Other Lessons Learned

No other Lessons Learned have currently been identified for this requirement.

7. Software Assurance

SWE-054 - Corrective Action for Inconsistencies
4.1.6 The project manager shall identify, initiate corrective actions, and track until closure inconsistencies among requirements, project plans, and software products. 

7.1 Tasking for Software Assurance

From NASA-STD-8739.8B

1. Monitor identified differences among requirements, project plans, and software products and confirm differences are addressed and corrective actions are tracked until closure.

7.2 Software Assurance Products

  • List of any inconsistencies identified among products, including risks and issues


    Objective Evidence

    • Document change management system results.
    • Software problem reporting or defect tracking system results.

    Objective evidence is an unbiased, documented fact showing that an activity was confirmed or performed by the software assurance/safety person(s). The evidence for confirmation of the activity can take any number of different forms, depending on the activity in the task. Examples are:

    • Observations, findings, issues, risks found by the SA/safety person and may be expressed in an audit or checklist record, email, memo or entry into a tracking system (e.g. Risk Log).
    • Meeting minutes with attendance lists or SA meeting notes or assessments of the activities and recorded in the project repository.
    • Status report, email or memo containing statements that confirmation has been performed with date (a checklist of confirmations could be used to record when each confirmation has been done!).
    • Signatures on SA reviewed or witnessed products or activities, or
    • Status report, email or memo containing a short summary of information gained by performing the activity. Some examples of using a “short summary” as objective evidence of a confirmation are:
      • To confirm that: “IV&V Program Execution exists”, the summary might be: IV&V Plan is in draft state. It is expected to be complete by (some date).
      • To confirm that: “Traceability between software requirements and hazards with SW contributions exists”, the summary might be x% of the hazards with software contributions are traced to the requirements.
    • The specific products listed in the Introduction of 8.16 are also objective evidence as well as the examples listed above.

7.3 Metrics

  • # of Corrective Actions (CAs) raised by SA vs. total #
    • Attributes (Type, Severity, # of days Open, Life Cycle Phase Found)
    • State (Open, In work, Closed)
    • Trends of CA Open vs. Closures over time
  • # of incorrect, missing, and incomplete requirements (i.e., # of requirements issues) vs. # of requirements issues resolved
  • Trend the # of inconsistencies or corrective actions identified, and # closed.
  • # of software work product Non-Conformances identified by life cycle phase over time
  • Trend of Open vs. Closed Non-Conformances over time

See also Topic 8.18 - SA Suggested Metrics

7.4 Guidance

1. Analyze the different project planning documents and confirm that all inconsistencies are documented and addressed. Items that are often overlooked include some of the following:

  • A set of requirements with estimated effort too great for the allotted time and effort planned in the schedule (Also, budget and schedule mismatch with requirements)
  • Items planned for implementation or purchase too late for the "need" date for integration
  • Development schedules not aligned with "need" dates
  • Assurance activities planned that don't align with project phases or activities
  • Requirements needing expertise not planned or coordination with another group or development that has not been planned

Any inconsistencies identified during the SA review of project plans, requirements, and software products should be documented along with suggested corrective actions, submitted to the project management, and tracked to closure. As the project progresses, inconsistencies among different project products should be kept in mind, particularly during reviews and meetings, and continually identified and addressed as needed

2. Software assurance should analyze all proposed changes for impacts, looking closely at any impacts the change may have on any of the software related to safety or security. The analysis should also consider whether there will be any impacts on existing interfaces or the use of any COTS, GOTS, MOTS, or reused software in the system and whether the change will impact any future maintenance effort. Any identified risks should be brought up to discuss the approval/rejection of the change.

Examples:

  • Ensure that the software requirements are met by the design and code.
  • Ensure that the planned activities in the project software plans are followed and completed
  • Ensure that the software products meet the requirements, conform to the software plans, and meet the acceptance criteria 
  • Ensure that the software code meets the software requirements and does not contain features and capabilities that are not included in the requirements, make sure that all software features and capabilities are tested.

3. Confirm:

  • That the project tracks the changes

Software assurance will check to see that any changes that are submitted are properly documented and tracked through all the states of resolution (investigation, acceptance/rejection, implementation, test, closure) in the project tracking system.

  • That the changes are approved and documented before implementation

Software assurance should track the changes from their submission to their closure or rejection. Initially, SA should confirm that all changes follow the change management process that the project has established. Initially, the change will be documented and submitted to the authorizing CCB for consideration. The authorizing CCB (which will include a software assurance person) will evaluate any changes for impacts.

If the software is safety-critical, the responsible software assurance personnel will perform a software safety change analysis to evaluate whether the proposed change could invoke a hazardous state, affect a control for a hazard, condition, or state, increase the likelihood or severity of a hazardous state, adversely affect safety-critical software, or change the safety criticality of an existing software element.  Keep in mind that changes to the hardware or the software can impact the overall system’s safety and while the focus is on software changes, the software also needs to be aware of changes to the hardware that may impact how software controls, monitors and analyzes inputs from that hardware.  Hardware and software changes can alter the role of software from non-safety-critical to safety-critical or change the severity from moderate to critical. 

Some other considerations for the evaluation of changes:

      • Is the change an error correction or a new requirement?
      • Will the change fix the problem without major changes to other areas?
      • If major changes to other areas are needed, are they specified, and is this change really necessary?
      • If the change is a requirements change, has the new requirement been approved?
      • How much effort will be required to implement the change?
      • If there is an impact on safety or reliability, are there additional changes that need to be made in those areas? Note: If there is a conflict between safety and security, safety changes have priority.

When all the impacts are considered, the CCB votes on acceptance/rejection. Software assurance is a voting member of the CCB.  Software assurance verifies that the decision is recorded and is acceptable, defined as:

      • When the resolution is to “accept as is”, verify that the impact of that resolution on quality, safety, reliability, and security is compatible with the Project’s risk posture and is compliant with NPR 7150.2 and other Center and Agency requirements for risk.
      • When the resolution is a change to the SW, the change will sufficiently address the problem and will not impact quality, safety, reliability, security, and compliance with NPR 7150.2; the change will not introduce new or exacerbate other, discrepancies or problems.
      • In either case, the presence of other instances of the same kind of discrepancy/problem has been sought out and, if detected, addressed accordingly.
      • Verify that appropriate software severity levels are assigned and maintained.
      • Assure any risk associated with the change is added to the Project/facility risk management system and is addressed, as needed, in safety, reliability, or other risk systems
  • That the implementation of the changes is complete

Software assurance will check to see if the implementation of the approved changes has been coded as per the change request. Check to see that any associated documentation changes are submitted/approved and/or made as needed (i.e., updates to requirements, design, test plans/procedures, etc.)

  • That the project tests the changes

Software assurance will check to see that the project test any of the code that has changed and runs a set of regression tests to see that the change has not caused a problem anywhere else in the software system. If the software is safety-critical, a full set of regression tests should be run to ensure that there was no impact on the safety-critical functions.

4. Confirm software changes are done in the software control process

Software assurance will check that the software control process has been followed throughout the handling of the submitted change and that the status of the change is  recorded and confirmed as closed

7.5 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook:

  • No labels