Maintaining item banks with the Rasch model: An example from wave optics

Item banks are generally considered the basis of a new generation of educational measurement. In combination with specialized software, they can facilitate the computerized assembling of multiple pre-equated test forms. However, for advantages of item banks to become fully realized it is important that the item banks store a relatively large number of valid test items. In this paper, we demonstrate how the Rasch model is used for integrating new items into an existing wave optics item bank. First, we identified and applied a set of criteria for selecting 18 linking items from our initial item bank. In order to integrate 12 newly developed items, we combined the 18 linking items with the 12 newly developed ones into one test and administered this test to 106 postinstruction physics students from 4 universities in Slovenia, Croatia, and Bosnia and Herzegovina. It was determined that the 12 new items measure the same construct as items from the initial item bank. In addition, all items exhibited good item fit, item reliability was excellent and person reliability was fair. The ratio of standard deviations of linking item difficulties for the new test and existing item bank amounted to 0.89 and correlation of these difficulties amounted to 0.93 which indicated good linking precision. We could conclude that good linking precision can be obtained if linking items are chosen based on the following set of criteria: number of items, item fit, range and spacing of item difficulties, content representativeness, position in test form and interuniversity DIF contrasts.

Figure 1

Line of best fit for cross plot of difficulties of common items from the initial and depositing study, as well as a 95% confidence band are shown as evidence that difficulties of common items are invariant within the limits of measurement error.

Figure 2

Cross plots for full-test-based vs subtest-based estimates of item difficulties for “old” and “new” items.

Figure 3

Wright map for the combined analysis. Common items’ labels begin with the letter “C.” New items are green and four items that were dropped from the linking set are shown in yellow. For example, the label “Item 16A” describes the functioning of Item 16A in the initial study and “Item 16A2” describes the functioning of the same item in the depositing study.