Cutting scores for tests developed by Ramsay Corporation are determined by use of a modification of Angoff’s procedure (Livingston & Zieky, 1982).

This procedure consists of use of a panel of job experts (persons who have knowledge about a job or subject area) led by a psychologist (familiar with psychometrics and test use). The panel is asked to think about a group of qualified workers. The panel then looks at each test item. In turn, each group member states the correct answer to the test item. The psychologist confirms or denies that the answer is correct. If the answer is incorrect, the group tries to determine why the incorrect answer is given. Then the first job expert states the estimated percent of qualified workers who would get that item correct. In turn, the other job experts state the percent of qualified workers they believe would answer the item correctly. When considering qualification, job experts are asked to include highly qualified persons, middle qualified persons, and even barely qualified persons but not those "below minimum acceptable performance." After they have heard the judgments of the other workers, each may leave his or her percentage as is or may adjust it up or down. When data are available for an item, the psychologist will indicate the percent passing in the largest group to whom the item was given. The total number of items on the test is reviewed. At this time any test items may be removed or altered.

The percentages rated by the job experts are then statistically analyzed as follows. Each job expert's judgments are totaled and then all totals are added together. That sum is divided by the number of raters and the number of test items. This number is called the Angoff. It is expressed as a decimal and on an easy test might be .90 and on a difficult test might be .50. The psychologist then multiplies the percentage times the number of items on the test. This number is called "Angoff times items." The psychologist then subtracts from "Angoff times items" 1.6 times the Standard Error of Measurement (Lyman, 1986). This provides for a margin of error because 90% of the time a person’s true score is 1.6 Standard Error of Measurement units from his or her obtained score.

Therefore, the resultant cutting score is a conservative estimate by knowledgeable job experts.

References:

Livingston, S. A., & Zieky, M. J. (1982). Passing scores: A manual for setting standards of performance on educational and occupational tests. Princeton, NJ: Educational Testing Service.

Lyman, H. B. (1986). Test scores and what they mean (4th ed.). Englewood Cliffs, NJ: Prentice Hall.