Verification and Validation

Michael Wales - NeedMoreHP@yahoo.com
EEL 5874 - Expert Systems and Knowledge Engineering

Version History:

Document	Date	Description
Verification and Validation V1.0	March 5, 2002	Initial Document

Table of Contents

• Verification Criteria
• Validation Criteria

The verification for the required ComputerAdvice system will focus on the application following the correct procedures, without any errors. The verification phase is not concerned about the correctness of the output. The following table lists the verification criteria for the system.

Required System - Command prompt system

1. The system shall be free from compilation errors (and warnings), runtime errors, and shall not throw any unhandled exceptions.
2. Given a sufficient budget, the system shall always be able to create a functional computer system. The components that must exist for the system to be functional are as follows: CPU, Case, Motherboard, RAM, HD, Optical, Mouse, KB, Floppy
3. The application shall prompt the user for all of the possible common computer applications listed below: Gaming, Video-editing, Surf the Internet, Software development, Audiophile, Home office, Multimedia server, Graphics development
4. The ComputerAdvice and JESS applications shall not throw any exceptions if the system is being operated properly.
5. The system shall prompt the user to enter a number for the computer system budget.
6. The system shall print out a computer system build sheet, and then exit without any errors or thrown exceptions.
7. The system shall meet all the requirements having the keyword "shall" in the requirments document.

For the required command prompt system, all of the above verification requirements need to be completed in full. In each and every test case for the application, all of the above requirements should be satisfied. If any requirement is not met during testing, the system will be examined and corrected, and then tested in full again.

Optional System - Web based system

1. Once the webserver is started, the application can constantly run and will not require any more administration to continue running.
2. The system should gracefully handle new client connections, and the server side application should be invisible to the client user. The system should start with an empty set of knowledge for each new connection; no knowledge should remain in memory from previous connections.
3. The system should be able to handle dropped connections, where the user terminated the client application without completely running through the application.

Since this part of the application is optional, not all of these verification criteria need to be met. If any of the optional system verification criteria fails during testing, an effort will be made to correct the problem. If the time required to fix the system will be large enough to affect the development and testing of the required system, the optional system will remain as-is.

The validation phase concentrates on examining the output of the system for correctness. The system will be investigated to make sure that the output is correct, but also that the reasoning is correct also. Validation also makes sure that then end product is what the end user expected.

The validation is the same for both the required system (command prompt based) and the optional system (web based). The systems only differ in how the user interacts with the system, the knowledge engine is identical for each system. The validation will likely occur with the command prompt system simply because it is faster than the web based system.

The validation methodology used will be a hybird of informal validation and validation by testing. Informal validation is where the developer of the system would meet with the expert and informally test the system. Validation by testing is where test cases are used to make sure that the system agrees with the expert's results. The will use a hybrid of these two processes because the problem my system trys to solve never has a real definite answer.

The validation process will be as follows:

1. The expert and I will create 6 test scenarios that represent a broad range of common system applications.
2. The expert will create an approximate build sheet for the system, and a good price for the user to budget on the entire system.
3. The system will be given the test scenarios and the budgeted amount for the system.
4. The expert will compare the build sheet that he created to the build sheet the system created. They will not match. The expert will evaluate the ComputerAdvice results, and give a rating such as: strongly agree, agree, acceptable, disagree, unacceptable.
5. The facts created by each test case will be documented, and the expert will verify that the reasoning used by the ComputerAdvice system is acceptable.