Item response theory evaluation of the Light and Spectroscopy Concept Inventory national data set

[This paper is part of the Focused Collection on Astronomy Education Research.] This paper presents the first item response theory (IRT) analysis of the national data set on introductory, general education, college-level astronomy teaching using the Light and Spectroscopy Concept Inventory (LSCI). We used the difference between students’ pre- and postinstruction IRT-estimated abilities as a measure of learning gain. This analysis provides deeper insights than prior publications both into the LSCI as an instrument and into the effectiveness of teaching and learning in introductory astronomy courses. Our IRT analysis supports the classical test theory findings of prior studies using the LSCI with this population. In particular, we found that students in classes that used active learning strategies at least 25% of the time had average IRT-estimated learning gains that were approximately 1 logit larger than students in classes that spent less time on active learning strategies. We also found that instructors who want their classes to achieve an improvement in abilities of average $\mathrm{\Delta }\theta =1$ logit must spend at least 25% of class time on active learning strategies. However, our analysis also powerfully illustrates the lack of insight into student learning that is revealed by looking at a single measure of learning gain, such as average $\mathrm{\Delta }\theta$. Educators and researchers should also examine the distributions of students’ abilities pre- and postinstruction in order to understand how many students actually achieved an improvement in their abilities and whether or not a majority of students have moved to postabilities significantly greater than the national average.

This article appears in the following collection:

Figure 1

Item 21 from the LSCI.

Figure 2

Items 18 and 19 from the LSCI.

Figure 3

Item 20 from the LSCI.

Figure 4

The distribution of pre- and postinstruction abilities for all 3205 students in the data set.

Figure 5

The distribution of IRT-estimated gains ($\mathrm{\Delta }\theta$) for all 3205 students in the data set.

Figure 6

The distribution of pre- and postinstruction abilities for the 2178 students in high-IAS classes with at least 25 students.

Figure 7

The distribution of pre- and postinstruction abilities for the 363 students in low-IAS classes with at least 25 students.

Figure 8

The distribution of IRT-estimated gains ($\mathrm{\Delta }\theta$) for the 2178 students in high-IAS classes and the 363 students in low-IAS classes.

Figure 9

$\mathrm{\Delta }\theta$ versus $g$ for all 3205 students in the LSCI national data set.

Figure 10

Average $\mathrm{\Delta }\theta$ versus $⟨g⟩$ for classes with at least 25 students in the LSCI national data set.

Figure 11

Average $\mathrm{\Delta }\theta$ versus IAS for classes with at least 25 students in the LSCI national data set.

Figure 12

The pre- and postability histogram for class 29.

Figure 13

The pre- and postability histograms for class 19.

Figure 14

The pre- and postability histograms for class 1.

Figure 15

The pre- and postability histogram for class 1.

Figure 16

The pre- and postability histogram for class 2.

Figure 17

The pre- and postability histogram for class 3.

Figure 18

The pre- and postability histogram for class 4.

Figure 19

The pre- and postability histogram for class 5.

Figure 20

The pre- and postability histogram for class 6.

Figure 21

The pre- and postability histogram for class 7.

Figure 22

The pre- and postability histogram for class 8.

Figure 23

The pre- and postability histogram for class 9.

Figure 24

The pre- and postability histograms for class 10.

Figure 25

The pre- and postability histograms for class 11.

Figure 26

The pre- and postability histograms for class 12.

Figure 27

The pre- and postability histograms for class 13.

Figure 28

The pre- and postability histograms for class 14.

Figure 29

The pre- and postability histograms for class 15.

Figure 30

The pre- and postability histograms for class 16.

Figure 31

The pre- and postability histograms for class 17.

Figure 32

The pre- and postability histograms for class 18.

Figure 33

The pre- and postability histograms for class 19.

Figure 34

The pre- and postability histogram for class 20.

Figure 35

The pre- and postability histogram for class 21.

Figure 36

The pre- and postability histogram for class 22.

Figure 37

The pre- and postability histogram for class 23.

Figure 38

The pre- and postability histogram for class 24.

Figure 39

The pre- and postability histogram for class 25.

Figure 40

The pre- and postability histogram for class 26.

Figure 41

The pre- and postability histogram for class 27.

Figure 42

The pre- and postability histogram for class 28.

Figure 43

The pre- and postability histogram for class 29.