Exploring physics students’ engagement with online instructional videos in an introductory mechanics course

The advent of new educational technologies has stimulated interest in using online videos to deliver content in university courses. We examined student engagement with 78 online videos that we created and were incorporated into a one-semester flipped introductory mechanics course at the Georgia Institute of Technology. We found that students were more engaged with videos that supported laboratory activities than with videos that presented lecture content. In particular, the percentage of students accessing laboratory videos was consistently greater than 80% throughout the semester. On the other hand, the percentage of students accessing lecture videos dropped to less than 40% by the end of the term. Moreover, the fraction of students accessing the entirety of a video decreases when videos become longer in length, and this trend is more prominent for the lecture videos than the laboratory videos. The results suggest that students may access videos based on perceived value: students appear to consider the laboratory videos as essential for successfully completing the laboratories while they appear to consider the lecture videos as something more akin to supplemental material. In this study, we also found that there was little correlation between student engagement with the videos and their incoming background. There was also little correlation found between student engagement with the videos and their performance in the course. An examination of the in-video content suggests that students engaged more with concrete information that is explicitly required for assignment completion (e.g., actions required to complete laboratory work, or formulas or mathematical expressions needed to solve particular problems) and less with content that is considered more conceptual in nature. It was also found that students’ in-video accesses usually increased toward the embedded interaction points. However, students did not necessarily access the follow-up discussion of these interaction points. The results of the study suggest ways in which instructors may revise courses to better support student learning. For example, external intervention that helps students see the value of accessing videos may be required in order for this resource to be put to more effective use. In addition, students may benefit more from a clicker question that reiterates important concepts within the question itself, rather than a clicker question that leaves some important concepts to be addressed only in the discussion afterwards.

Figure 1

The “accessing trajectory” of a single student. The dashed diagonal line represents where video time and real time are identical.

Figure 2

Time series indicating the number of times student 3 accessed a given second in video 15 during that student’s 5th accessing session ($A_{ijkt}$ with $i=3$, $j=15$, $k=5$). In this session, student 3 accesses most of the video once while accessing 519–567 s multiple times.

Figure 3

Complementary cumulative distribution function $C(V)$ showing the fraction of students accessing more than a certain proportion of videos in each group. Students are much more likely to access most or all of the laboratory videos in comparison to the lecture-oriented videos.

Figure 4

Fraction of accessing students (${\mathrm{FS}}_j$) for each video. The lecture-oriented videos are represented in gray. The laboratory videos are represented in red. No videos were assigned in week 9, week 14, week 16, and week 17. Laboratory videos specific for the 1st, 2nd, 3rd, and 4th lab were assigned in weeks 1 and 2, week 4, week 6, and week 8, respectively. The 5th lab was a “choose your own adventure” lab, in which students were expected to take advantage of what they learned in the course to explore any kind of motion that was of interest to them. Therefore, no laboratory videos specific for this lab were provided. Weeks 1 and 2 also included supplemental lab videos concerning specific tools used in all lab activities instead of specific lab activities. Major topic breakdown for the lecture-oriented videos is indicated on the figure. The figure shows that students reduced their accessing of lecture oriented videos as time progressed while maintaining their access for laboratory videos.

Figure 5

Video length vs fraction of students accessing more than 99% of the given video (${\mathrm{FSE}}_j$). Each data point represents one single video in the course. The lecture-oriented videos are color coded based on different topics, with black, orange, yellow, green, blue, and purple representing the 1st to 6th topics presented in Table 2, respectively. The trend lines for all lecture-oriented videos combined and all laboratory videos combined are $y=-0.000447x+0.758$ and $y=-0.000167x+0.655$, respectively. The figure shows that the fraction of students accessing the entirety of a video decreases when videos become longer in length. Moreover, this trend is more prominent for the lecture-oriented videos than the laboratory videos.

Figure 6

Clickstream analysis of video 15. This video, entitled “Creating a Computer Model of Constant Velocity Motion,” was accessed by 148 unique students in total. The black line represents the number of times each particular point of a given video was accessed by students (i.e., $\sum _{i=1}^{N_s}{\mathrm{TA}}_{ijt}$, where $j=15$). The blue line represents the percentage of unique students who has ever paused within each 5 s time window. If multiple pauses were made by the same student within a 5 s window, this student was counted only once in the given window. Since we are interested in student-generated pauses, the computer-generated pauses both at the interaction points and the very end of the video have been taken out. In order to help identify peaks, pauses are not plotted for all the 5 s time windows. Instead, only cases for which the percentage of students pausing is greater than or equal to 2 median absolute deviations [27] above the median value of all cases (i.e., $2\sigma +\text{median}$), which in this case corresponds to 7.4%, are shown in the figure. The interaction points in the video are indicated by the vertical dashed line. Places where “actions” are demonstrated in the video are indicated by the black arrows. The figure shows that student engagement increased during these action points. A description of the action each arrow represented is included in Table 6.

Figure 7

Clickstream analysis of video 11. This video, entitled “Newton’s second law”, was accessed by 153 unique students. The black line represents the number of times each particular point of a given video was accessed by students (i.e., the value at in-video time $t$ was obtained by $\sum _{i=1}^{N_s}{\mathrm{TA}}_{ijt}$, where $j=11$). The blue line represents the percentage of unique students that have ever paused within each 5 s time window. To help identify common pause peaks, only cases for which the percentage of students pausing is greater than or equal to 2 median absolute deviations above the median value of all 5 s windows ($2\sigma +\text{median}$, which in this case corresponds to 7.2%) were shown in the figure. Since we are interested in student-generated pauses, automatic pauses that occurred at the very end of the video have been taken out.