Correlations between student connectivity and academic performance: A pandemic follow-up

Social network analysis (SNA) has been gaining traction as a technique for quantitatively studying student collaboration. We analyze networks, constructed from student self-reports of collaboration on homework assignments, in two courses from the University of Colorado Boulder and one course from the Colorado School of Mines. All three courses occurred during the COVID-19 pandemic, which allows for a comparison between the course at the Colorado School of Mines (in a fully remote format) with results from a previous pre-pandemic study of student collaboration at the Colorado School of Mines (in an in-person format), as well as comparison between the Mines results with the two University of Colorado courses (in a hybrid format). We compute nodal centrality measures and calculate the correlation between student centrality and performance. Results varied widely between each of the courses studied. The course at the Colorado School of Mines had strong correlations between many centrality measures and performance which matched the patterns seen in the pre-pandemic study. The courses at the University of Colorado Boulder showed weaker correlations, and one course showed nearly no correlations at all between students’ connectivity to their classmates and their performance. Taken together, the results from the trio of courses indicate that the context and environment in which the course is situated play a more important role in fostering a correlation between student collaboration and course performance than the format (remote, hybrid, in-person) of the course, a finding which has implications for the broader use of SNA within physics education research. Additionally, we conducted a short study on the effect that missing nodes may have on the correlations calculated from the measured networks, an analysis largely missing from the SNA literature within PER. This investigation showed that missing nodes tend to shift correlations towards zero, providing evidence that the statistically significant correlations measured in our networks are not spurious.

Figure 1

Example of a complex network. Here the blue dots denote nodes (or students) and lines between the notes represent links (or edges). Directionality of the link is denoted by the direction of the arrow on each link.

Figure 2

An isolated node with three inward links and two outward links. This node has in-strength $s_i=8$, out-strength $s_o=4$, and net-strength $s_n=4$.

Figure 3

Collaboration network from the Fall 2020 course at Mines. This network does not show six disconnected nodes representing students for whom no collaborations were reported. Nodes are colored based on the outward directed closeness centrality $c^{\mathrm{Co}}$.

Figure 4

Collaboration network from the Fall 2020 course at CU Boulder. This network does not show 16 disconnected nodes representing students for whom no collaborations were reported. Nodes are colored based on the outward directed closeness centrality $c^{\mathrm{Co}}$.

Figure 5

The three components, representing three isolated groups of students, of the network from the Spring 2021 thermal physics course at CU Boulder. There were also 16 fully disconnected students who do not appear in the networks above. Nodes are colored based on the outward directed closeness centrality $c^{\mathrm{Co}}$. (a) First component of the collaboration network from the Spring 2021 thermal physics course at CU Boulder, (b) Second component of the collaboration network from the Spring 2021 thermal physics course at CU Boulder, and (c) Third component of the collaboration network from the Spring 2021 thermal physics course at CU Boulder.

Figure 6

Correlations between nodal centrality measures and student performance in three pandemic-affected courses. Correlations significant at the $p<0.05$ level are indicated with filled markers. The math methods course at Mines had no exams. (a) Fall 2020 Math Methods at Mines, (b) Fall 2020 Thermal Physics at CU Boulder, and (c) Spring 2021 Thermal Physics at CU Boulder.

Figure 7

Distribution of edge weights in the Fall 2020 CU Boulder thermal physics course and in simulated networks. The height of the bars is the log of the ratio of the number of links of a particular weight to square of the number of nodes, i.e., the total number of ordered pairs of nodes.

Figure 8

Distributions of reciprocities for each of the four random network methods developed in this study. For each method, 1000 random networks were created.

Figure 9

Histogram of correlations between grades and inward harmonic centrality, $c^{\text{Hi}}$, in networks created by dropping nodes from the full Mines network. The red lines in each plot represent the correlation between grades and inward harmonic centrality measured in the original Mines network. The green and black lines represent the mean and median correlation of the ensemble of reduced networks. The blue bars represent the histogram of correlations calculated from the networks in the ensemble. (a) Histogram of correlations in reduced networks created by preferentially dropping nodes with low grades from the Mines network, and (b) Histogram of correlations in reduced networks created by preferentially dropping nodes with low centrality from the Mines network.

Figure 10

Histogram of correlations between grades and inward harmonic centrality, $c^{\text{Hi}}$, in networks simulated with the DD method and 36% of nodes removed based on the nodes’ centrality. (a) Preferentially dropping nodes with high centrality, and (b) Preferentially dropping nodes with low centrality.

Figure 11

Scatter plots of homework scores versus outward directed closeness and harmonic centrality. (a) Homework Score versus outward directed closeness centrality in the fall 2020 Mines course, (b) Homework score versus outward directed closeness centrality in the fall 2020 CU Boulder course, (c) Homework score versus outward directed closeness centrality in the spring 2021 CU Boulder course, (d) Homework score versus outward directed harmonic centrality in the fall 2020 Mines course, (e) Homework score versus outward directed harmonic centrality in the fall 2020 CU Boulder course, and (f) Homework score versus outward directed harmonic centrality in the spring 2021 CU Boulder course.