Item Response Theory - Essay Example

Summary Perhaps the cornerstone of applied psychology is psychological testing. While various elements, such as legal guidelines or applications, have changed, psychological tests have remained largely stable over the past few decades. Indeed, there are a number of cornerstone tests that while amended, have remained largely the same. Although new tests occasionally emerge, the foundations for the development of these tests remain highly consistent. Still, the psychometric basis of tests has noticeably changed. The foundational approach had long been classical test theory (CTT). This approach constituted base test elements, yet has increasingly been replaced or supplemented by item response theory (IRT). Item response theory has increased been implemented in the development of standardized tests as this approach is believed to contribute more readily to solving practical problems. Scope of Item Response Theory Applications Item response theory is now the foundational theoretical model for a number of psychological tests. One mode of tests implementing ITR is computerized adaptive testing. This mode of testing collects the individual’s responses as a means of measuring their optimal potential. IRT principles are implemented in selecting testing criteria for the particular individual, as well as comparing scores among a variety of subject criteria. This testing approach has been witnessed within the Armed Serviced Vocational Aptitude Battery, the Scholastic Aptitude Test (SAT), and the Graduate Record Examination (GRE). IRT has also been implemented in a variety of intelligence tests. For instance, IRT principles underline the Different Ability Scales, the Woodcock-Johnson Psycho-Educational Battery, and the current version of the Stanford-Binet. Furthermore, IRT has been implemented in personality trait measurements, including mood, situational evaluation, cognitive elements, and behavioral dispositions (Reise & Waller, 1990). History of IRT While item response theory constitutes an overarching theoretical approach, there are differing elements within this model. Notably, there are two lines of IRT development. Generally, the original elements of IRT is linked to Lord and Novick’s (1968) textbook Theories of Mental Test Scores. Within this book there are four chapters on IRT, all of them written by Allan Birmbaum. Within these articulations there is also an essay by Lord (1953), as well as three United States Air Force technical reports. While other theorists did not readily consider the Air Force reports at the time, they were still included (Hambleton & Swaminathan, 1985). This textbook would come to be recognized as a cornerstone contribution to psychometric methods. This is attributed to a multiplicity of reasons. While previous methods were partly unscientific and lacked rigor, the methods contained in this text implemented a unified statistic test theory. To an extent the methods espoused in this text are linked to Gulliksen’s exposition of CTT in Theory of Mental Tests; still, this new texts greatly expanded the statistical analysis. The textbook is also highly significant as it is connected to testing methods. Fred Lord, the predominant author, had been a long-time worker at Educational Testing Services. This institution was responsible for a variety of high volume testing practices. The textbook was connected and adopted to leading and emerging scholars in psychometric methods. Perhaps most prominent among these was R. Darrell Bock, a professor at the University of North Carolina. Professor Brock implemented IRT principles in the development of his own models. Specifically, Brock has concerned with the implementation of algorithms within the parameters of IRT models. Bock had a number of students and followers, including David Thissen, Eiji Muraki, Richard Gibbons, and Robert Mislevy. These individuals developed computer programs and estimation methods; these productions include TestFact, Bilog, and Parscale. In collaboration with the Murray Aitken, Bock designed marginal maximum to estimate specific parameters. This approach is now widely used within IRT practices (Embretson & Reise, 2000). Item Response Theory models Assumptions: There are three main assumptions in item response theory models. 1. The item characteristics curve (ICC) has a specific form. This form is significant as it describes the relation between changes in trail level to that of the probability of specified response. For dichotomous items, the ICC regresses the probability at trait level. In terms of polytomous items – for instance rating scales – the ICC regresses the response probability (Embretson & Reise, 2000). 2. Unidimensionality An assumption of IRT models is that only one ability is measured by an item set. This poses challenges as the assumption cannot be met because cognitive, personality, and test-taking factors always impact test performance. For instance, motivation level or test anxiety can have a significant impact on the test. In order to meet the unidimensionality assumption it’s necessary for the implementation of a dominant component. This is referred to as the ability measured by the test; this is not inherent or unchangeable, however. Because of a variety of factors this score will change over time (Hambleton et al, 1991). 3. Local Independence Local independence refers to the process where external environmental concerns remain constants, the examinees’ response to any pair of items are statistically independent. Specifically, after examinees’ abilities are considered no direct relationship exists between their responses. This is significant as it indicates that the abilities specific to the model are the only factors that influence response to test items. These abilities are referred to as the complete latent space. When unidimenstionality holds, the complete latent space only consists of one ability. To state the definition of local independence more formally, let θ be complete set of abilities assumed to influence the performance of an examinee on the test. Let U_i be the response of a randomly chosen examinee to item I (i=1,2,3,………,n). let P(U_(i )θ) denote the probability of the response of a randomly chosen examinee with ability θ ;P(U_i=1θ) denotes the probability of a correct response, and P(U_i=0θ) denotes the probability of an incorrect response (Hambleton et al, 1991). Advantages of Item Response Theory Models: There are a number of advantages to the implementation of the IRT model. a. An IRT trait level estimate can be computed from any item, as long as the properties are known. b. Test behavior and item properties are directly correlated. c. The independent variables, trait level and item properties, can be estimated separately and this estimation does not require additional data. This is notably significant in comparison to the CTT model, as the later does not have these properties. Popular Models in Item Response Theory The item characteristic curve (ICC) is a mathematical expression that links the probability of success on an item measured to its characteristic. Within this mode of recognition its understood that there are infinite possibilities for IRT models; still, there are only a limited amount in practical use. A primary distinction within these models occurs along specific parameters. Model choice is user contingent, yet this decision incorporates verifiable data through observed test results. The three most popularly implemented unidimensional IRT models are the one, two, and three –parameter logistic models. These are regularly implemented for dichotomous item response data. One-parameter Logistic Model (Rasch Model) This model is perhaps the most widely implemented IRT model. The item characteristic curves for the one-parameter logistic model are given by the equation P_i(θ) =e^((θ-b_i))/(1+e^((θ-b_i)) ) i = 1,2,……….,n [2,1] Where P_i(θ) is the probability that a randomly chosen examinee with ability θ answers item I correctly, b_i is the item I difficulty parameter, n is the number of the item in the test e is a transcendental number with value 2.718 (correct to three decimals). The Read More