Validity of Force Concept Inventory evaluated by students’ explanations and confirmation using modified item response curve

In this study, we evaluate the validity of question 29 (Q.29) in the Force Concept Inventory (FCI), which requires an understanding of atmospheric pressure. At least in some Japanese physics classes, the rate of the correct answer to Q.29 is reduced between the pretest and post-test. To examine this deterioration of a student’s response, we analyzed 429 explanations written by 358 respondents about the force described in the question. Many learners, who were not aware of atmospheric force, chose the correct answer. On the other hand, many of the wrong respondents had learned about air pressure but did not understand it. As a result, postinstruction learners tended to have a worse rate of correct response. This made Q.29 less sensitive to a Japanese student’s ability. This characteristic feature of Q.29 is visually confirmed by a modified item response curve, which represents correlation between the response to one item and the total score of the other items.

Figure 1

Distribution of the FCI scores.

Figure 2

Score distribution of (B) choosers and (D) choosers with different reasoning.

Figure 3

Item response curve of Q.29. (a) Normal IRC using the score of 30 Items. (b) Modified IRC using the score of 29 items other than that of Q.29 as a proxy of the students’ ability.

Figure 4

Schematically simplified image of MIRC. Misconception oriented image of Q.29. The areas (A), (B), and (D) represent the percentage of responses. The rare responses of (C) and (E) are ignored. The broken lines are qualitative estimates.

Figure 5

Comparison of MIRCs obtained in Hyogo and Hiroshima with two MIRCs obtained in the U.S. The MIRCs in the U.S. were transformed from normal IRCs using a score distribution similar to Fig. 1 (solid lines) and the uniform distribution (broken line).