Description of Technical Test Validation Procedure

A. Job Analysis

The client provides a company job description.  We observe the job and put the description into Ramsay Corporation's format of Job Summary, Work Performed, and Consequences of Errors.  Ten job experts (supervisors or incumbents) edit the description to ensure that it is up-to-date and reflects the job activities.  The top ten job tasks are listed and ranked in importance.


B. Completion of Maintenance Activity List

The ten job experts rate the job with respect to importance of job activities and percent of time spent on job activities.  Ramsay Corporation evaluates the interrater agreement.


C. Editing of Knowledge and Skills Areas

We provide a list of knowledge and skills areas for the job.  The job experts edit the list to ensure that the important areas are represented.  They rank the knowledge and skills areas with respect to relative importance and the number of items to be included in each area.


D. Selection of Test Items

Job experts are assigned to work in pairs.  They are given applicable sections of Ramsay Corporation's Database.  Each test item in the database includes Difficulty (percent of persons getting the item right) and Discrimination Index (correlation of getting the item right with the area or total test score).  Job experts usually have 3-8 times as many items as they wish to select.  They are able to select items reflecting the level of sophistication, types of equipment, and level of difficulty required.


E. Edit , Compose, and Print Draft Tests

Ramsay Corporation leaves the first day's meeting with the selected test items designated.  We then edit the test and obtain 2 drafts of the test.  The second draft is photocopied and taken to the job experts.


F. Angoff Procedures To Set Cutting Scores and Review Final Test

Five to ten job experts review the test item-by-item.  They select the correct answer, they are told the keyed answer, and they are asked what percent of qualified candidates would pass each item.  They are told the percent passing the item in the largest group to whom the item has been given.  This is also their final review of the total test before final printing.


G. Writing of Content Validation Report

Ramsay Corporation writes a content-related validation report in the format suggested by the Uniform Guidelines on Employee Selection Procedures.  The company reviews a draft.  The final report detailing the activities undertaken is then bound and provided to the client.