It is important to remember that functional requirements are the ‘what’ and nonfunctional requirements are the ‘how’. So, the testing of functional requirements is the verification that the software is executing actions as it should, while nonfunctional testing helps verify that customer expectations are being met.
An early metric for gauging the success of meeting functional requirements would be the results of unit testing. The next step would be looking at the number of issues captured while dry-running verifications. Finally, the number of issues found and resolved along with the number of requirements verified during formal verification are good metrics to track.
Performance requirements can be difficult to determine since many are domain and project-specific. They tend to be dependent on the software design as a whole including how it interacts with the hardware. Performance requirements are usually, but not always, related to the Quality attributes of the software system.
Some typical performance requirements involve the following:
- Peak demand processing i.e. transactions per second over some prescribed duration
- Sustained processing i.e. transactions per second over any time span
- Response time i.e. time to service an interrupt
- Storage capacity/utilization
- Sampling rates
- CPU utilization
- Memory capacity/utilization
At a minimum, these items should be evaluated and reported to the Project at all major milestone reviews as well as any major maintenance upgrades.