What do seniors remember from freshman physics?

We have given a group of 56 Massachusetts Institute of Technology (MIT) seniors who took mechanics as freshmen a written test similar to the final exam they took in their freshman course as well as the Mechanics Baseline Test (MBT) and the Colorado Learning Attitudes about Science Survey (CLASS). Students in majors unrelated to physics scored 60% lower on the written analytic part of the final than they would have as freshmen. The mean score of all participants on the MBT was insignificantly changed from their average on the posttest they took as freshmen. However, the students’ performance on 9 of the 26 MBT items (with 6 of the 9 involving graphical kinematics) represents a gain over their freshman posttest score (a normalized gain of about 70%), while their performance on the remaining 17 questions is best characterized as a loss of approximately 50% of the material learned in the freshman course. On multiple-choice questions covering advanced physics concepts, the mean score of the participants was about 50% lower than the average performance of freshmen. Although attitudinal survey results indicate that almost half the seniors feel the specific mechanics course content is unlikely to be useful to them, a significant majority (75%–85%) feel that physics does teach valuable problem solving skills, and an overwhelming majority believe that mechanics should remain a required course at MIT.

Figure 1

The solid line represents a pure gain curve because it has its pretest score axis intercept at 100%. The absolute value of the slope of the gain curve is equal to the average normalized gain. The dotted line is a pure loss curve because it has its pretest score intercept at 0%. The absolute value of the slope of the loss curve is equal to the average fraction of knowledge lost. The dashed line is a sample linear fit to a data set which is neither pure loss nor pure gain.

Figure 2

Score shift between the end of freshman mechanics and the end of senior year on analytic problems versus freshman performance. The dotted line is the boundary corresponding to a score of zero on the retest. The solid lines are fits to the group 1 data only (gray with floating intercept, black with $x$ intercept fixed at zero). The floating fit has $x$ intercept $-10.0\pm 11.9$, which is consistent with zero. The slope of the fixed fit indicates knowledge loss of $0.59\pm 0.04$ in the seven semesters since taking mechanics.

Figure 3

Score shift from the end of freshman mechanics to graduation on the MBT versus freshman posttest score. The solid line is a fit to the data for all groups of students, the dashed line fits the group 1 students only. The $x$ intercepts of the fits (70.0% and 60.0%, respectively) are clearly not consistent with zero or 100%.

Figure 4

Score shift from the end of freshman mechanics to graduation on subtest G of the MBT versus freshman posttest score on the same subtest. The solid lines are fits to the data for all groups (gray with floating intercept, black with $x$ intercept fixed at 9). The dashed lines are fits to the group 1 data (gray with floating intercept, black with $x$ intercept fixed at 9). The floating fit to the entire group has an $x$ intercept of $8.42\pm 0.31$, consistent with an $x$ intercept of 9. The fixed fit to the entire group implies a normalized gain of $0.68\pm 0.05$ over the 3.5 years since taking freshman mechanics.

Figure 5

Score shift from the end of freshman mechanics to graduation on subtest L of the MBT versus the score achieved on subtest L at the end of the freshman course. The solid line fits the data for all groups of students, the dashed line fits the group 1 data only.

Figure 6

Score shift from freshman posttest to senior retest on subtest L of the MBT versus shift during the freshman class. Solid fits are to the data for all groups (gray with floating intercept, black with $x$ intercept fixed at zero). The fit with floating intercept has $p=0.50$ for an intercept equal to zero. The dashed line fits the group 1 data only with $x$ intercept fixed at zero. The slopes of the fits with fixed intercepts are $-0.52\pm 0.09$ and $-0.48\pm 0.09$, respectively.

Figure 7

Histogram showing the number of students in group 1 with a given loss fraction on the analytic problems of the retest. The hatched areas are a stacked histogram of the students who self-reported full (white) or less than full (black) effort on this portion of the retest.

Figure 8

Plot of percentage loss on analytic questions versus deviation of final exam $z$ score from mean midterm $z$ score for group 3 students. The $y$ intercept of the linear fit is $35\%\pm 20\%$.

Figure 9

Frequency with which various topics taught in mechanics were circled as “useful to you since taking [mechanics]” by students on the demographic survey.

Figure 10

Histogram showing the distribution of the ratio of the square of the standard deviation observed in 35 four-problem “tests” (combinations of four MIT final exam problems) to the sum of the squared standard deviations of the four individual problems.