How an Examination is Developed
General Steps in the Development Process
These are general steps and the actual steps used for a specific examination may vary. Examinations play a vital role in the credentialing process – assuring minimal competence to practice a profession at the entry level. There are several general steps followed in the development process. These steps are:
- Job/Task Analysis
This process refers to the study of the elements of knowledge, skill and ability necessary for an individual to practice. It also refers to the determination of those tasks which are important in determining competent performance and are typically performed by job incumbents. - Test Specifications
This activity guides the test developers in constructing examinations which are consistent with the job analysis and ensuring that each form of the examination tests the same basic concepts. The test specifications serve as a “blueprint” for the examination. - Developing Objectively Scored Examinations
Question Development
This process refers to the actual writing of examination questions as well as the review of the questions from the perspectives of both conformance with sound testing practice and factual accuracy. Questions are written by subject matter experts who are licensed members of the profession for which the examination is being developed.
Assembling an Examination Form
After a pool or bank of acceptable test questions has been constructed, the pool serves as the basis for construction. To assemble forms of the examination, questions are selected which meet the specifications or “blueprint”.
Standard Setting
Equally important to the sound development of an examination form is the determination of a standard, or minimum passing score, used to compare and interpret the test scores. The minimum passing score is established by each professional board after considering recommendations made by the Bureau.
Pilot/PreTesting New Questions
Examinations may contain a small numbers of pretest unscored or “pilot questions”. The purpose of including pilot test questions within the examinations is to expand and improve the bank of questions from which future examination will be drawn. This is a common practice used by many national and state examination programs and is a critical step in ensuring the continued reliability and validity of these examinations. Pilot questions are not counted when computing test scores. Pilot questions are not identified. If the pilot questions were identified, many of the candidates would skip them and the results would not be valid. The development of a good examination requires accurate candidate response information for the pilot questions.
Timing the Examination
A time limit is established for the examination administration ensuring that the majority of the candidates will have ample time to complete the examination. In the event that pilot questions are included within the examination, additional time will be given for answering the pilot questions.