Value added or misattributed? A multi-institution study on the educational benefit of labs for reinforcing physics content

Instructional labs are widely seen as a unique, albeit expensive, way to teach scientific content. We measured the effectiveness of introductory lab courses at achieving this educational goal across nine different lab courses at three very different institutions. These institutions and courses encompassed a broad range of student populations and instructional styles. The nine courses studied had two key things in common: the labs aimed to reinforce the content presented in lectures, and the labs were optional. By comparing the performance of students who did and did not take the labs (with careful normalization for selection effects), we found universally and precisely no added value to learning course content from taking the labs as measured by course exam performance. This work should motivate institutions and departments to reexamine the goals and conduct of their lab courses, given their resource-intensive nature. We show why these results make sense when looking at the comparative mental processes of students involved in research and instructional labs, and offer alternative goals and instructional approaches that would make lab courses more educationally valuable.

Figure 1

The mean lab benefit [Eq. (2)] on the final exams in nine courses at three institutions. Error bars represent the standard error ($=\mathrm{standard}$ uncertainty in the mean). The dashed line indicates the hypothesis being tested (whether the mean lab benefit differs significantly from 0).

Figure 2

The mean conceptual lab benefit [Eq. (2)] for six courses at two institutions. Error bars represent the standard error (= standard uncertainty in the mean). The dashed line indicates the hypothesis being tested (whether the mean lab benefit differs significantly from 0).

Figure 3

The mean lab benefit [Eq. (2)] across midterms for six courses at two institutions for (a) the full exams and (b) the conceptual items only. Not all midterm exams had sufficient conceptual questions to be included in the analysis. Error bars represent the standard error (or standard uncertainty in the mean). The dashed line indicates the hypothesis being tested (whether the mean lab benefit differs significantly from 0).

Figure 4

Two exam questions about electric field and force from one of the institutions. Question 12 was categorized as lab related while question 13 was categorized as not lab related.

Figure 5

The list of learning objectives from a lab about electric field and force at the same institution as that used in Fig. 4.

Figure 6

An exam question about momentum from one of the institutions. It was categorized as conceptual.

Figure 7

An exam question about momentum from one of the institutions. It was categorized as calculational.