"Test Development" by Patrick & Harley - A review

This Post is Authored by Sree Krishna (During migration the authors name could not be migrated)
There are different resources, which help us to understand the concepts of Testing or the Test Development process. I feel, that all these materials cannot be used in isolation to understand the concept. Different authors emphasize on different aspects and hence we should use more than one reading material to understand the concept. A single point reference is never advisable in these cases. As a matter of fact, some of them might not be suitable at all and might further mislead or confuse the audience.

I would suggest a few reference materials to facilitate a better understanding of the science/concept of Assessments, from beginners to advanced learners. Here are a few of them:



URLs
http://www.ukcle.ac.uk/resources/trns/mcqs/nine.html
http://www.msu.edu/dept/soweb/writitem.html
http://pareonline.net/getvn.asp?v=5&n=2

Books
Psychological Testing by Paul Kline
Psychological Testing by Anastasi
Item writing by Thomas M Haladyna


I did go through "Test Development" by Patrick and Harley and the first thought that struck my mind is "it does not cover all the aspects of Test Development in detail", "this cannot be a training document for a Test Development Team". It can be, at the best, a book to understand the basic elementary aspects of Test Development.

My apprehensions about recommending this book as a training document:

a) Does not cover the methodology of understanding/analyzing the requirements and mapping the requirements to assessments
b) Does not cover the types of assessments (so, the differences are not clearly understood)
c) Explanations are not elaborate – For example Bloom's taxonomy
d) Reasons for using Objective Assessments and their limitations - not mentioned at all
e) Guidelines for Item writing – does not emphasize the importance / significance of the line items mentioned, for example: "Research studies have proven that the negative words will have difficulty in understanding the meaning of negatively phrased items. A negatively-stated item does require a test taker to switch his or her mind set from that of looking for the best answer to that of locating the most definite non-answer" or "Use of specific determiners such as always, never, totally, absolutely, only, and completely in all the distracters or only in the right answers serve as flags to the test taker"
f) Examples provided for Item Writing Guidelines are not appropriate for MeritTrac or rather dated. The learning audience may not relate the course content back to the work place scenario.
g) Many questions are left unanswered. For example, why are Norms used, is not answered. Similarly Explanations are not satisfactory for z scores. The concepts need more clarity in terms of explanation and examples.
h) Explanations on Reliability and Validity are brief and sketchy.

I feel that Item Creators/Reviewers can use this material for reference purpose, after they have undergone a formal/extensive and interactive training on Assessment. And the textual reference will definitely enhance the acquired learning gained from the training sessions.