|View Our Catalog||
Adults: Critical Issues in Testing and Evaluation|
FUNDAMENTAL PROBLEMS IN TEST CONSTRUCTION
Since the great majority of school-based tests, college-entrance tests, and licensure tests are of the multiple-choice variety, it is now appropriate to list some of the fundamental problems they pose for this population:
1. Some items in such tests contain item bias in the sense that they presume experience or present some content that would not be equally accessible to a deaf person, such as items related to music or rhyme, or (especially in the days prior to closed captioning), television, movies, and other staples of “hearing” culture.
2. Many such tests include some items that are embedded in figurative or colloquial English. A deaf person who has attained some level of mastery of the English language through reading and writing, using strategies less tied to phonological processing due to restricted or no auditory input, would not be able to access the intent of the item simply because it was worded by a hearing test developer who assumed a knowledge of colloquial English. Often there is not sufficient redundancy within the item to allow the test-taker to discern the meaning from context. This problem is particularly relevant when such language is not one of the constructs that is being tested, yet it prevents the test-taker from accessing the meaning of the item.
3. Many such tests provide inadequate context in the “stem” of the item (that portion of the item which describes the situation on which several subsequent multiple-choice items are based); research indicates that deaf readers require a fuller context before being able to make a selection (citation?).
4. Some items use words that have multiple meanings in cases where the deaf or hard of hearing test-taker might only have had access to a single meaning for the same word. If an alternate meaning, other than the one that the deaf or hard of hearing person knows, was intended by the test developer, then the deaf or hard of hearing person will get that item wrong. Related to this problem are words on the test that are used in ways that are not frequently used in other non-testing contexts.
5. Some items use multiple embedded dependent clauses in item wording. Deaf and hard of hearing candidates can easily become lost in that complexity and miss the intent of the question itself. Test-makers sometimes do such embedding in order to increase item difficulty or get more information into less space or time, even though such structures are not the specific structures that are being measured by the test.
Other problems could be identified, but even one error resulting from any of the above five flaws could result in a candidate not meeting the cut-score (sometimes referred to as “cut-off” score) and mean the difference between graduation or non-graduation, or licensure or non-licensure, for a deaf or hard of hearing person. When one aggregates these flaws, the likelihood of the deaf or hard of person having a problem with a test because of its embeddedness in English increases severalfold.