Framework and implementation for improving physics essential skills via computer-based practice: Vector math

We propose a framework for improving accuracy, fluency, and retention of basic skills essential for solving problems relevant to STEM introductory courses, and implement the framework for the case of basic vector math skills over several semesters in an introductory physics course. Using an iterative development process, the framework begins with a careful identification of target skills and the study of specific student difficulties with these skills. It then employs computer-based instruction, immediate feedback, mastery grading, and well-researched principles from cognitive psychology such as interleaved training sequences and distributed practice. We implemented this with more than 1500 students over 2 semesters. Students completed the mastery practice for an average of about $13\min /\mathrm{week}$, for a total of about 2–3 h for the whole semester. Results reveal large ($>1\mathrm{SD}$) pretest to post-test gains in accuracy in vector skills, even compared to a control group, and these gains were retained at least 2 months after practice. We also find evidence of improved fluency, student satisfaction, and that awarding regular course credit results in higher participation and higher learning gains than awarding extra credit. In all, we find that simple computer-based mastery practice is an effective and efficient way to improve a set of basic and essential skills for introductory physics.

Figure 1

Experimental design used in this study. Experiment 3 was conducted 11 weeks after the post test in experiment 1.

Figure 2

Example practice questions for addition (top right), components vasic (top left), and cross product direction (bottom) skills. Current student scores for each skill are recorded on the “progress wheels” to the left. Incorrect answer (red circle “$x$”) results in the score returning to zero for that skill.

Figure 3

Example explanation for the topic of simple vector components.

Figure 4

Experiment 1 completion rates for all 12 training units.

Figure 5

Histogram of total time spent in training for units 4–12. Only students who attempted all of these units are included. The dotted line represents the median.

Figure 6

Box plots of time spent training for units 4–12. All student attempts for each of the units are included. Only the middle two quartiles are shown.

Figure 7

Distribution of pre- and post-test scores for experiment 1.

Figure 8

Comparison of scores preinstruction and postinstruction, with and without essential skills practice. Skills to the left of the dotted line were explicitly taught in the first semester course, while skills to the right received no explicit instruction. Error bars are 1 S.E.

Figure 9

Longitudinal student assessment performance. A subset of experiment 1 students (pretest and post-test scores in dashed bars) completed a delayed post test (first solid bar) during the second semester physics course. A sample of students from the same second semester course who had not previously received training also completed the post test (second solid bar) at the same time.

Figure 10

Pretest and post-test scores for experiments 1 and 2, which varied whether the essential skills were assigned for course credit or extra credit.