Investigating the use of mastery-style online homework exercises in introductory algebra-based mechanics in a controlled clinical study

Homework in introductory physics represents an important part of a student’s learning experience; therefore, choosing the manner in which homework is presented merits investigation. We performed three rounds of clinical trials comparing the effects of mastery-style homework vs traditional-style homework with students in both algebra-based and calculus-based introductory mechanics. Results indicate a benefit from mastery-style over traditional-style homework, principally for weaker students who are less familiar with the material being covered and on questions that are nearer transfer to the study materials.

Figure 1

Normalized average scores on each of the twelve questions on the post-test used in the first clinical trial. Average scores with standard errors for students in the mastery-homework condition are plotted on the vertical axis, and average scores with standard errors for students in the traditional-homework condition are plotted on the horizontal axis. Points appearing above the diagonal line indicate questions on which the students in the mastery-homework condition outperformed students in the traditional-homework condition. Inset: Normalized total average scores on the post-test with standard errors are shown for the two homework conditions. Dots on the inset graph represent individual students’ total scores. Compare Fig. 4 from Ref. [14].

Figure 2

Performance of students in the first clinical trial, with nearer- and farther-transfer questions analyzed separately. (a) Performance of students on those post-test questions which were judged to be nearer-transfer to the homework problems. As seen on the inset graph, the difference in total scores between the mastery and traditional homework groups was judged to be significant with $p=0.009$ ($p=0.018$ two-tail $t$-test) and a Cohen’s d effect size of 0.9. (b) Performance of students on those post-test questions which were judged to be farther-transfer to the homework problems. As seen on the inset graph, the difference in total scores between the mastery and traditional homework groups was not significant.

Figure 3

Performance of students in the first clinical trial with students in the top and bottom half of each homework condition group analyzed separately. This median split was performed using the students’ final semester course scores. Dots on the inset graphs show individual student post-test totals. (a) Performance of students in the bottom half of each homework condition group as measured by their final semester course scores. The difference in post-test total scores is significant with $p<0.05$ on a one-tail $t$-test. (b) Performance of students in the top half of each homework condition group as measured by their final semester course scores. The difference in posttest total scores is not significant.

Figure 4

Performance of students in the second and third clinical trials. Average scores with standard errors for students in the mastery-homework condition are plotted on the vertical axis, and average scores with standard errors for students in the traditional-homework condition are plotted on the horizontal axis. Average total scores for the students in each homework condition are shown in the inset. Dots on the inset graphs show individual student post-test totals. (a) Performance of students in the second clinical trial. This trial involved students in calculus-based physics and covered the topic of friction, using the same materials as the first clinical trial. This trial was run before the first midterm in the course. The difference in total scores is not significant. (b) Performance of students in the third clinical trial. This trial involved students in algebrabased physics and covered the topic of Newton’s laws. The clinical trial was run before the course’s comprehensive final exam. The difference in total scores is not significant.

Figure 5

Figure for friction post-test problem 1.

Figure 6

Figure for friction post-test problem 2.

Figure 7

Figure for friction post-test problem 3.

Figure 8

Figure for friction post-test problems 4–5.

Figure 9

Figure for friction post-test problem 6.

Figure 10

Figure for friction post-test problem 7.

Figure 11

Figure for friction post-test problems 8–9.

Figure 12

Figure for friction post-test problems 10–11.

Figure 13

Figure for friction post-test problem 12.

Figure 14

Figure for Newton’s laws post-test problem 1.

Figure 15

Figure for Newton’s laws post-test problem 2.

Figure 16

Figure for Newton’s laws post-test problems 4–6.

Figure 17

Figure for Newton’s Laws post-test Problem 7.

Figure 18

Figure for Newton’s Laws post-test Problem 8.

Figure 19

Figure for Newton’s Laws post-test Problem 9.

Figure 20

Figure for Newton’s Laws post-test Problem 10.