Improving performance in quantum mechanics with explicit incentives to correct mistakes

An earlier investigation found that the performance of advanced students in a quantum mechanics course did not automatically improve from midterm to final exam on identical problems even when they were provided the correct solutions and their own graded exams. Here, we describe a study, which extended over four years, in which upper-level undergraduate students in a quantum physics course were given four identical problems in both the midterm exam and final exam. Approximately half of the students were given explicit incentives to correct their mistakes in the midterm exam. In particular, they could get back up to 50% of the points lost on each midterm exam problem. The solutions to the midterm exam problems were provided to all students in both groups but those who corrected their mistakes were provided the solution after they submitted their corrections to the instructor. The performance on the same problems on the final exam suggests that students who were given incentives to correct their mistakes significantly outperformed those who were not given an incentive. The incentive to correct the mistakes had greater impact on the final exam performance of students who had not performed well on the midterm exam.

Figure 1

Average gain (defined as the difference between posttest and pretest score) vs pretest score for each student on the four questions repeated from the pretest to the posttest in the comparison group (A) and incentivized group (B). Red-filled triangles are for each student from the comparison group and the blue-filled squares are for each student in the incentivized group in which students received an explicit grade incentive to correct their own mistakes in the pretest before answering the same questions on the posttest. Students whose scores improved are above the horizontal axis; students whose performance deteriorated are below the horizontal axis.

Figure 2

Average pretest vs posttest score for each student on the four questions repeated from the pretest to the posttest in the comparison group and incentivized group. Red-filled triangles are used for each student from the comparison group and the blue-filled squares are for each student in the incentivized group. The histograms along each axis show the distribution of the pretest and posttest scores for the comparison group and the incentived group. A closer look at the histograms suggests that pre-intervention scores are similar for the comparison and incentivized group. However, the incentivized group’s posttest distribution has shifted out of the lower region, and is peaked near the high performance cutoff.

Figure 3

An example of a pretest and posttest solution pair for a student from the comparison group which demonstrates a typical deterioration on the expectation value problem. The pretest and posttest scores are 100% and 50% for invoking, 89% and 33% for applying, and 95% and 42% for the overall scores, respectively. While the pretest solution is almost perfect, the posttest solution shows student difficulties including confusion between the discrete spectrum of the eigenstates of $\widehat{Q}$ and the continuous spectrum of position eigenstates and difficulty with the treatment of the operator $\widehat{Q}$.

Figure 4

An example of a pretest and posttest solution pair for a student from the comparison group for the measurement problem. The pretest and posttest scores are 36% and 8% for invoking, 13% and 17% for applying, and 24% and 13% for the overall scores, respectively. The student struggles with this problem in the pretest. In the posttest, no improvement is demonstrated.

Figure 5

An example of a pretest and posttest solution pair for a student from the comparison group which demonstrates a typical deterioration for the momentum problem. The pretest and posttest scores are 89% and 0% for invoking, 100% and 0% for applying, and 95% and 0% for the overall scores, respectively. The pretest solution is nearly perfect, but in the posttest, no relevant knowledge is displayed.

Figure 6

An example of a pretest and posttest solution pair for a student from the comparison group which demonstrates a typical deterioration for the harmonic oscillator problem. The pretest and posttest scores are 100% and 40% for invoking, 83% and 0% for applying, and 92% and 20% for the overall scores, respectively. The student’s pretest performance is good, except for some minor errors with the coefficients when dealing with the ladder operators but the posttest solution demonstrates that this proficiency has deteriorated.

Figure 7

An example of a pretest and posttest solution pair for a student from the incentivized group, which demonstrates improvement in student understanding for the expectation value problem. The pretest and posttest scores are 54% and 100% for invoking, 33% and 100% for applying, and 44% and 100% for the overall scores, respectively. While the pretest solution shows difficulty with the expansion of a general state in terms of the eigenstates of the operator $\widehat{Q}$ and confusion between the eigenvalue of $\widehat{Q}$ and the probability amplitude for measuring $Q$, the posttest shows excellent use of Dirac notation to solve the problem.

Figure 8

An example of a pretest and posttest solution pair for a student from the incentivized group which demonstrates improvement in student understanding for the measurement problem. The pretest and posttest scores are 0% and 100% for invoking, 25% and 100% for applying, and 13% and 100% for the overall scores, respectively. The pretest solution shows that the student incorrectly inserts $x=a/2$ in the expression for the energy eigenfunction instead of calculating the probability of measuring energy. The posttest solution is essentially perfect.

Figure 9

An example of a pretest and posttest solution pair for a student from the incentivized group which demonstrates improvement in student understanding for the momentum problem. The pretest and posttest scores are 20% and 100% for invoking, 0% and 89% for applying, and 10% and 95% for the overall scores, respectively. The pretest shows exploratory derivations focused on expectation value of momentum that has nothing to do with the correct solution. The posttest solution is a succinct derivation of the desired result.

Figure 10

An example of a pretest and posttest solution pair for a student from the incentivized group which demonstrates improvement in student understanding for the harmonic oscillator problem. The pretest and posttest scores are 33% and 100% for invoking, 0% and 100% for applying, and 17% and 100% for the overall scores, respectively. The pretest shows that the student struggles and eventually abandons the problem, but in the posttest, the student demonstrates proficient use of the ladder operators to solve the problem.