Mastery-style homework exercises in introductory physics courses: Implementation matters

Encouraged by positive clinical results at the University of Illinois, mastery-style homework was integrated into a large semester-long preparatory physics course via an online homework system that used narrated animated video solutions as correctives. This paper discusses the impact and evolution of the homework in its first two years. The first iteration revealed that students were frustrated and did not engage with the system in an effective way. Intending to reduce that frustration and quell negative behavior, the mastery requirement was relaxed, transfer between versions was reduced, and the addition of a direct discussion with students about the homework were implemented in its second year. The results showed that details of implementation can substantially affect students’ behavior; large and statistically significant effects were observed as a reduction in frustration (with self-identified “frustrated” students dropping from 60% in 2014 to 30% in 2015) and improvement in performance (average student mastery rate of 59% to 69%).

Figure 1

Online mastery schematic shown of two example levels within week 7: Most weeks had 3–5 levels, with 4 versions (A–D) of problems and solutions each. Students were randomly assigned to version A, B, C, or D for their first try and had access to solutions after grading the version, then were either allowed to move on to the next level (if mastered) or call the next version of the same level (if unmastered). Calling a new version of the same level removed access to solutions and problems of the previous version.

Figure 2

Normalized frustration histogram for 2014 tested over time: Students responded to the question, “Which statement best reflects your experience working through the mastery homework?”

Figure 3

Normalized frustration histograms from (a) Fall 2014 and (b) Fall 2015: Students responded to the question, “Which statement best reflects your experience working through the mastery homework?”

Figure 4

Solution viewing: The fraction of students who watched over half of the solution videos to their missed problems, separated by levels in two unchanged weeks from 2014 to 2015.

Figure 5

Student progression through unchanged weeks’ levels in 2014 and 2015, (a) week 3 and (b) week 7: Progression is shown as the fraction of students who had mastered each level by try.

Figure 6

Average student mastery rates through semester by week: Both axes are the average fraction of levels mastered in each week, in 2014 and 2015. Points above the line indicate improved mastery rates for 2015 over 2014. This plot includes only students who attempted all levels, or all levels except one.

Figure 7

Diagram provided in the prompt for version A of week 7, level 1 problems.

Figure 8

Answer choices given for question 1 on version A of week 7, level 1 problems.

Figure 9

Diagram provided for question 5 on version A of week 7, level 1 problems.

Figure 10

Diagram provided in the prompt for version B of week 7, level 1 problems.

Figure 11

Answer choices given for question 1 on version B of week 7, level 1 problems.

Figure 12

Diagram provided in the prompt for version C of week 7, level 1 problems.

Figure 13

Answer choices given for question 1 on version C of week 7, level 1 problems.

Figure 14

Diagram provided in the prompt for version D of week 7, level 1 problems.

Figure 15

Answer choices given for question 1 on version D of week 7, level 1 problems.