Deeper look at question categories, concepts, and context covered: Modified module analysis of quantum mechanics concept assessment

Concept inventories are commonly used tools in physics education research (PER) for evaluating teaching methods. Student responses to concept inventories have been studied using classical test theoretical methods such as factor analysis and network methods such as module analysis. The results of these studies have been used to evaluate the instruments and to find aggregate patterns that may be used to infer how students’ think about the concepts on the inventory. In this work, we apply modified model analysis using partial correlations, a network based method using partial correlations, to evaluate the Quantum Mechanics Concept Assessment (QMCA). The QMCA is designed to measure student understanding of upper-division quantum mechanics in both the spatial wave function and the spins contexts. This represents the first time a network-based analysis has been applied to an upper-division concept inventory. The modules we found are related to the structure of the instrument. There were two broad classes of modules: those that connected responses (either both correct or both incorrect) to isomorphic pairs questions written to probe the same concept in the spatial wave function and the spin contexts. The second class of modules were explain pairs, where the first question asked about a concept and the second question asked students to explain their choice, again either both correct or both incorrect. This structure resembles the structure of the Force Concept Inventory where there are a common correct and a common incorrect pair for many concepts, compared to the Force and Motion Conceptual Evaluation, which has larger and more varied communities reflecting a smaller number of topics.

Figure 1

The empirical cumulative distribution function displays the percent correct on the QMCA on the $x$ axis and the percent of the sample with that score or less on the $y$ axis. The data have a slight right skew, but are approximately normal.

Figure 2

Example of isomorphic questions. In the spatial wave function context students work with the energy eigenstates in the infinite square well. In the spins context, students work with spin-$1/2$ and spin-1 systems. The states are both written with identical coefficients and the questions ask for the same conceptual information.

Figure 3

Communities of responses commonly selected together by students identified by the MMA-P analysis of the QMCA instrument, including the correct (green nodes) and incorrect (pink nodes) responses. Edges represent the partial correlation, given by the label, between the responses and only edges with correlation of small effect size or greater are shown. There are no intercommunity edges; each set connects nodes in a single community. Cyan edges connect responses to isomorphic items, blue edges connect responses to explain pairs, and black edges are none of these. The thickness of the edges is proportional to the number of times in all the bootstrapped networks that the two responses were chosen together. The communities consist of nodes that clustered together in at least 80% of the bootstrapped networks.