Design and validation of the Quantum Mechanics Conceptual Survey

The Quantum Mechanics Conceptual Survey (QMCS) is a 12-question survey of students’ conceptual understanding of quantum mechanics. It is intended to be used to measure the relative effectiveness of different instructional methods in modern physics courses. In this paper, we describe the design and validation of the survey, a process that included observations of students, a review of previous literature and textbooks and syllabi, faculty and student interviews, and statistical analysis. We also discuss issues in the development of specific questions, which may be useful both for instructors who wish to use the QMCS in their classes and for researchers who wish to conduct further research of student understanding of quantum mechanics. The QMCS has been most thoroughly tested in, and is most appropriate for assessment of (as a posttest only), sophomore-level modern physics courses. We also describe testing with students in junior quantum courses and graduate quantum courses, from which we conclude that the QMCS may be appropriate for assessing junior quantum courses, but is not appropriate for assessing graduate courses. One surprising result of our faculty interviews is a lack of faculty consensus on what topics should be taught in modern physics, which has made designing a test that is valued by a majority of physics faculty more difficult than expected.

Figure 1

(Color) Average pretest scores in different modern physics courses for common questions (gray/light) and Version 2.0 questions (red/dark). Solid colored bars indicate courses taught with traditional instruction and cross-hatched bars indicate reformed courses. Error bars show standard error on the mean. The two dashed horizontal lines show the average score that would be expected if students answered using random guessing for the common questions (gray/light) and Version 2.0 (red/dark). The chart includes some courses in which no pretest was given in order to compare more easily to Fig. 2.

Figure 2

(Color) Average posttest scores in different courses for common questions (gray/light) and Version 2.0 questions (red/dark). Solid colored bars indicate courses taught with traditional instruction and cross-hatched bars indicate reformed courses. Error bars show standard error on the mean. All scores are post-test scores except for Grad QM CU, which is from a pre-test for an introductory graduate quantum course at CU, and Grad mixed, which is from a survey given over the summer to graduate students from different universities who may or may not have taken a graduate quantum course.

Figure 3

(Color) Scores on individual questions in Version 2.0 in modern physics classes. This graph combines data from all five modern physics classes in which all Version 2.0 questions were given. The blue bars on the right show the minimum and maximum average posttest score for individual courses.

Figure 4

Question 11, taken from the QMVI, tests student understanding of the relationship between wave function and probability density.

Figure 5

Question 12 tests student understanding of the wave-particle duality.

Figure 6

An alternative version of question 4, taken from Knight. This version does not work well because too many students do not notice the word “trajectory.”

Figure 7

The original version of question 10, taken from the QMVI. In the current version this question has been adapted in order to make it easier, by including only answers with larger magnitude on the left side. The correct answer is A.

Figure 8

(Color) Responses to the original slanted well question shown in Fig. 7 for students in four modern physics classes and for graduate students.

Figure 9

Two questions designed to elicit the belief that reflection and transmission are due to a range of energies, adapted from Ref. [30]. The correct answer to both questions is C.