Analysis of student engagement in an online annotation system in the context of a flipped introductory physics class

We discuss student participation in an online social annotation forum over two semesters of a flipped, introductory physics course at Harvard University. We find that students who engage in high-level discussion online, especially by providing answers to their peers’ questions, make more gains in conceptual understanding than students who do not. This is true regardless of students’ physics background. We find that we can steer online interaction towards more productive and engaging discussion by seeding the discussion and managing the size of the sections. Seeded sections produce higher quality annotations and a greater proportion of generative threads than unseeded sections. Larger sections produce longer threads; however, beyond a certain section size, the quality of the discussion decreases.

Figure 1

Examples of annotations, scores, and justifications for scoring.

Figure 2

Box and whisker plots depicting the distributions of annotation quality scores for students in each of the two semesters. The ends of the box are the upper and lower quartiles; the median is marked by the vertical line inside the box; the whiskers are the two lines outside the box that extend to the highest and lowest observations and, therefore, show the range.

Figure 3

Distribution of annotation types.

Figure 4

Distribution of thread types.

Figure 5

Average quality of annotations in unseeded versus seeded sections (left) and in unseeded versus seeded threads (right).

Figure 6

Percentage of thread types in unseeded versus seeded sections.

Figure 7

Average thread length and average annotation quality as a function of section size.