Comment on “ Benefits of completing homework for students with different aptitudes in an introductory electricity and magnetism course ”

In a recent study, Kontur, de La Harpe, and Terry [1] carried out an important study examining the differential benefits of doing more homework on different levels of students. Their surprising results, which suggest that doing homework does not benefit lower aptitude physics students, prompted us to examine the correlation between homework and exam grades in our own courses. In contrast to their findings, we see a correlation between homework and exam performance for all our students, with it being clearest for the lower aptitude. This emphasizes the need to examine questions about the benefit of homework carefully, recognizing the possible differences within and across different institutional populations and courses; a sentiment also conveyed by Kontur, de La Harpe, and Terry. Here we present data from two University of British Columbia (UBC) first-year physics courses. UBC is a large, relatively selective, public research university, and these courses are similar to those at many comparable institutions. One course is the introductory calculus-based electricity and magnetism course P102 (N 1⁄4 514) taken by essentially all science and engineering students, quite similar to the course Kontur, de La Harpe, and Terry analyze. The second course is algebra-based introductory mechanics P100 (N 1⁄4 744) taken primarily by a diverse group of students that did not take physics 12 in high school (60% science, 15% arts, 25% other majors). We include the latter here for comparison of the overall correlation between homework and exam scores, and in particular to see if there was a strong dependence on the exam format. The algebra-based mechanics course P100 has open-book-and-notes midterm and final exams, and so students have similar resources available as for the homework, while the electricity and magnetism course P102 has closed-book exams in which only one sheet with formulas is allowed. Otherwise, the homework and grading was the same between the two courses, with homework counting for 10% of the course grade. Unlike the sample of Kontur, de La Harpe, and Terry, these courses are taken by less than half the UBC student population. For the analysis of our data, we use a similar method as Kontur and co-workers. We use the Mastering Physics overall score as a measure for homework completion. In our courses, the default grading policy of the Mastering Physics software was modified to a more lenient due date policy (−10% for each day late with a maximum 50% penalty). For the exam score, we use the combined grade of midterm tests and the final examination, weighted as in the course grade of 1=4 and 3=4, respectively, for the electricity and magnetism course P102. The respective weights for midterms and final examination were 1=3 and 2=3, respectively, for the algebra-based introductory mechanics P100. We find a much stronger correlation between exam and homework performance for both courses than what was reported by Kontur, de La Harpe, and Terry. Figures 1(a) and 1(b) show the data for all students. The trends are similar in the two courses, but the correlation between homework and exams is stronger in the case of P102 (r 1⁄4 0.28 versus 0.18). Note that P100 has open-book examines, but, perhaps somewhat surprisingly, lower correlation. Possible reasons are discussed below. In Figs. 2(a)–2(d) we show the data from P102 broken down by aptitude quartile, as in Fig. 1 in the study of Kontur, de La Harpe, and Terry. Our aptitude breakdown is based on the students’ overall GPA at the end of term 1, just before starting P102. There is a strong correlation between this term 1 GPA and P102 exam performance, with an r of 0.50. This is a slightly different aptitude calculation than Corresponding author. rieger@phas.ubc.ca

In a recent study, Kontur, de La Harpe, and Terry [1] carried out an important study examining the differential benefits of doing more homework on different levels of students.Their surprising results, which suggest that doing homework does not benefit lower aptitude physics students, prompted us to examine the correlation between homework and exam grades in our own courses.In contrast to their findings, we see a correlation between homework and exam performance for all our students, with it being clearest for the lower aptitude.This emphasizes the need to examine questions about the benefit of homework carefully, recognizing the possible differences within and across different institutional populations and courses; a sentiment also conveyed by Kontur, de La Harpe, and Terry.
Here we present data from two University of British Columbia (UBC) first-year physics courses.UBC is a large, relatively selective, public research university, and these courses are similar to those at many comparable institutions.One course is the introductory calculus-based electricity and magnetism course P102 (N ¼ 514) taken by essentially all science and engineering students, quite similar to the course Kontur, de La Harpe, and Terry analyze.The second course is algebra-based introductory mechanics P100 (N ¼ 744) taken primarily by a diverse group of students that did not take physics 12 in high school (60% science, 15% arts, 25% other majors).We include the latter here for comparison of the overall correlation between homework and exam scores, and in particular to see if there was a strong dependence on the exam format.The algebra-based mechanics course P100 has open-book-and-notes midterm and final exams, and so students have similar resources available as for the homework, while the electricity and magnetism course P102 has closed-book exams in which only one sheet with formulas is allowed.Otherwise, the homework and grading was the same between the two courses, with homework counting for 10% of the course grade.Unlike the sample of Kontur, de La Harpe, and Terry, these courses are taken by less than half the UBC student population.
For the analysis of our data, we use a similar method as Kontur and co-workers.We use the Mastering Physics overall score as a measure for homework completion.In our courses, the default grading policy of the Mastering Physics software was modified to a more lenient due date policy (−10% for each day late with a maximum 50% penalty).For the exam score, we use the combined grade of midterm tests and the final examination, weighted as in the course grade of 1=4 and 3=4, respectively, for the electricity and magnetism course P102.The respective weights for midterms and final examination were 1=3 and 2=3, respectively, for the algebra-based introductory mechanics P100.
We find a much stronger correlation between exam and homework performance for both courses than what was reported by Kontur, de La Harpe, and Terry.Figures 1(a) and 1(b) show the data for all students.The trends are similar in the two courses, but the correlation between homework and exams is stronger in the case of P102 (r 2 ¼ 0.28 versus 0.18).Note that P100 has open-book examines, but, perhaps somewhat surprisingly, lower correlation.Possible reasons are discussed below.
In Figs.2(a)-2(d) we show the data from P102 broken down by aptitude quartile, as in Fig. 1 in the study of Kontur, de La Harpe, and Terry.Our aptitude breakdown is based on the students' overall GPA at the end of term 1, just before starting P102.There is a strong correlation between this term 1 GPA and P102 exam performance, with an r 2 of 0.50.This is a slightly different aptitude calculation than the GPA from three prerequisite courses used by Kontur and co-workers, but there is a substantial overlap of the respective courses involved and the r 2 's are essentially identical (0.50 versus 0.52).
In contrast to Kontur, de La Harpe, and Terry, we see a correlation between homework and exam scores only for the lowest aptitude students.The r 2 values are largely meaningless for the three higher aptitude quartiles, because such a high percentage of those students have nearly perfect scores on the homework.It is also interesting to see how the distributions of both exam and homework scores narrow for the higher aptitude students.Both features of the distributions are different from what is seen in the data of Kontur, de La Harpe, and Terry, indicating that different analyses may be needed for different populations.
Our data indicate that doing more homework is correlated with better exam performance when we consider the entire population of students.In contrast to the findings of Kontur and co-workers that lower aptitude students do not benefit from doing more homework, this correlation would imply the opposite for our students.
We offer some speculations as to the reasons for these observed differences.First, our homework tasks and exams could be more closely aligned than in those of Kontur, de La Harpe, and Terry.In P102 a slightly modified Mastering Physics problem was on each exam to emphasize the value of the homework.In P100, the exam questions were less similar to the homework questions than for 102, but carefully chosen to probe the same learning objectives.In both cases, the homework assignments are designed to practice specific skills that students are expected to learn and demonstrate on an exam.Each homework question and task serves a purpose that is aligned with a stated learning goal for the course.There is a large body of prior work from the learning sciences showing that practicing particular skills with feedback leads to the improvement in those skills (see, for example, Ambrose et al. [2]).Our homework is perhaps more carefully chosen than in other courses to provide the desired targeted practice and feedback.
Second, and perhaps most likely, the different contexts and student populations may be important.All United States Air Force Academy (USAFA) students have to take physics, and they have a far more regimented existence than our students, and this may impact their studying strategies and learning.For example, time might be a more precious resource for weaker students at USAFA and, hence, time for completing homework is invested at the expense of other performance-boosting strategies.Finally, the categorization of students by aptitude based on GPA might not be working in similar ways between USAFA's and our student population, although the correlations between courses are very similar in the two populations.
In summary, we find interesting differences between our data and the results presented by Kontur and co-workers, emphasizing the need for caution when drawing general conclusions about homework-exam correlations.It seems that in our courses, all students benefit from doing homework.We hope this additional data will further discussion of the important issue raised by Kontur, de La Harpe, and Terry as to how and why and for whom doing homework is beneficial.

FIG. 1 .
FIG. 1. Correlation between exam performance and Mastering Physics homework score (MP score) in two physics undergraduate courses at UBC.(a) Fig. 1(a) (left) The data for all students (N ¼ 744) in the algebra-based introductory mechanics course P100 and (b) Fig. 1(b) (right) shows the data for all students (N ¼ 514) in the calculus-based introductory electricity and magnetism course P102.The shaded areas indicate 95% confidence intervals.The median homework score is 9.9 in the case of P100 and 9.8 in the case of P102.The high scores are due to the use of extra problems (for bonus marks) in both courses to encourage practicing problem solving.