Educational commitment and social networking: The power of informal networks

The lack of an engaging pedagogy and the highly competitive atmosphere in introductory science courses tend to discourage students from pursuing science, technology, engineering, and mathematics (STEM) majors. Once in a STEM field, academic and social integration has been long thought to be important for students' persistence. Yet, it is rarely investigated. In particular, the relative impact of in-class and out-of-class interactions remains an open issue. Here, we demonstrate that, surprisingly, for students whose grades fall in the"middle of the pack,"the out-of-class network is the most significant predictor of persistence. To do so, we use logistic regression combined with Akaike's information criterion to assess in- and out-of-class networks, grades, and other factors. For students with grades at the very top (and bottom), final grade, unsurprisingly, is the best predictor of persistence---these students are likely already committed (or simply restricted from continuing) so they persist (or drop out). For intermediate grades, though, only out-of-class closeness---a measure of one's immersion in the network---helps predict persistence. This does not negate the need for in-class ties. However, it suggests that, in this cohort, only students that get past the convenient in-class interactions and start forming strong bonds outside of class are or become committed to their studies. Since many students are lost through attrition, our results suggest practical routes for increasing students' persistence in STEM majors.


I. INTRODUCTION
From industry to government to academia, attracting and retaining STEM (science, technology, engineering, and mathematics) majors is recognized as a key element of the 21 st century knowledge economy [1][2][3]. One approach to attracting and retaining students is to improve the overall educational experience, which requires understanding the immersion of students into the academic and social system of an institution. For instance, it has been noted that insufficient interactions with others, as well as a lack of compatibility with the social values of the institution, lead to a low commitment to the university [4]. Ultimately, this affects one's decision about whether to drop out. For students who just started their education and have not yet formed connections in the community, particularly those who commute to college, the classroom might be the only place where connecting with others happens. However, while both social and academic involvement seem to be essential for persistence, research on the effect of students' networks within the university is rare [5][6][7].
The importance of the classroom experience in introductory courses as a means for improving students' persistence should not be underestimated. Nearly half of first-time students who leave a university by the end of the freshman year never come back to college [8]. In a recent study, Zwolak et al. investigate academic and social experiences of students in an introductory physics class- * j.p.zwolak@gmail.com room via social network analysis (SNA) [7]. Their analysis of networks of self-reported, in-class interactions reveals that students with higher number of initiated or received interactions (as measured by directed degrees) and higher overall embeddedness within the network (as measured by closeness) at the end of the first semester were more likely to persist, i.e., to enroll in a second course in the sequence. Out of all the examined indices, closeness was the most correlated, giving up to 75 % chance of correctly predicting persistence. This agrees with earlier work that found closeness to be positively associated with students' perceptions of team effectiveness and performance, as well as on their attitudes and grades [9].
In a seminal paper, Tinto noted that "the manner in which social and academic involvements (integration) shape learning and persistence will vary . . . for different students inside and outside the classroom. . . . [N]etwork analysis and/or social mapping of student interaction patterns . . . will shed important light on how interactions across the academic and social geography of a campus shape the educational opportunity structure of campus life and, in turn, both student learning and persistence" [10]. In other words, when dealing with networks of peer-to-peer interactions, it is important to consider the environment that fosters their evolution. The inclass networks emerge from individuals who are located in the same time and space (i.e., the classroom during class-time) and may even be encouraged to work together (e.g., through group activities). In such a conducive environment where face-to-face interactions take place on a regular basis, one expects a robust network to develop. Such networks can be thought of as "networks of conve-FIG. 1. Model of persistence. We expand the model of persistence proposed in Ref. [7] to include frequency of the interactions and out-of-class networks. The categories of factors that may affect students' persistence are individual attributes, classroom context, as well as in-and out-of-class community. nience".
Out-of-class networks, on the other hand, are made up of individuals who typically choose to work together. They have to put much more effort into organizing and managing their interactions, be it in person or through some other media (e.g., messengers, text messages, online discussion boards or web-based office suites that allow for collaborating with others in real-time). Moreover, the out-of-class network can include not only students enrolled in the course, but also peers who took similar course in the past, friends enrolled in a different section of the same course, faculty that a student is comfortable interacting with or family members. Thus, the out-of-class networks should be thought of as "networks of choice".
In their analysis, Zwolak et al. use students' embeddedness within networks that include only the presence/absence and the direction of interactions, but not their frequency (i.e., they consider directed binary networks) [7]. We expand their model of persistence to more fully capture the multilevel character of the network data and to incorporate the out-of-class interactions, see Fig. 1. It has been noted that patterns of relations among students might explain outcomes, such as persistence or performance, over and above the attributes of either the individuals or the environment [11]. Our goal is to build an understanding of how students' proximity to others within the in-and out-of-class networks affects -or at least correlates with -their decision about whether to continue in the introductory physics course sequence.
Moreover, prior studies found that the lack of an engaging pedagogy that promotes active participation and the highly competitive atmosphere in introductory science courses tend to discourage students from pursuing STEM majors [12]. Over the past two decades, active learning strategies have become more common in the classroom [13][14][15][16][17]. The number, and presumably qual-ity, of interactions between peers is much higher in these settings than in traditional, lecture-based courses (see, e.g., Table IV in Ref. [7] and Table I in Ref. [18]. The relative effect of in-and out-of-class interactions, though, has yet to be addressed. We focus on the issue of students' persistence within an introductory Modeling Instruction (MI) physics sequence. MI is an approach that strongly emphasizes active learning and can be employed in any subject. The course is interaction-driven at both the small and large group levels. There are many ways for students to be involved and make connections -from group activities to group lab reports and, in this course, one group exam to the so-called "board meetings" [19]. Due of the interactive nature of the course, one expects the in-class network to develop quickly and to include all students enrolled in the course, as well as the instructional staff. Moreover, the nurturing collaborative environment of the MI classroom should promote the culture of working with peers also outside of class. Thus, the MI classroom is an important setting for determining the relevance of the networks of convenience and networks of choice on persistence.

The SNA survey
We collected social network data using a pencil and paper survey developed for this purpose. The survey was given five times throughout the semester. Every 2 to 3 weeks students were asked two questions regarding inand out-of-class interactions with their peers. For the in-class part, students were presented with a randomized list of names of all classmates, as well as the instructional staff. To simplify the survey completion, they could use two-digit codes randomly assigned to each person instead of full names. To capture the significance of students' inclass interactions, we presented them with a table where, based on repeatability of a given interaction, they were supposed to write names (or codes) of their peers in relevant columns (see Appendix S1 in Supplemental Material (SM) for a template of the social network survey). Below question about out-of-class networks was a blank space where students could provide the relevant information.   about the students' enrollment and teaching staff information for each section. There were three instructors teaching the course (denoted in Table I as A, B, and C), all physics education researchers. The instructors were accompanied by learning assistants (LAs) and teaching assistants (TAs), i.e., graduate students in physics education and high-achieving undergraduate students who took the MI course before, respectively. The TAs and LAs, as well as their numbers, varied between sections. Demographic information, i.e., gender, ethnicity, and academic plan were available for all students (see Table II for gender and ethnicity and Table S1 in the SM for academic plan). Between the four sections there were a total of 8 reported ethnic groups (including "choose not to report ethnicity"). Since multiple groups had three students or less, instead of considering each ethnicity as a separate category we combined them into two groups: minority and non-minority. Similarly, for academic plan: we had 32 different majors, with 19 options having less than four students. Thus, we avoid excessive fine graining by grouping majors together into 4 distinct categories (basic science, engineering, science other, other, see Table  S1 in SM for details). We found no significant difference between sections and the population as a whole in terms of gender (chi-square test: χ 2 (4) = 7.42, P = 0.12), ethnicity (chi-square test: χ 2 (4) = 5.81, P = 0.21) and academic plan (chi-square test: χ 2 (12) = 17.8, P = 0.12). The final grades data were available for all sections (N = 273) and the midterm scores were avail- able only for two sections from Fall 2015 (N = 144; two students in section A and one student in section B were missing midterm exam scores). The Kruskal-Wallis rank sum test showed no statistically significant difference in final grades between sections (χ 2 (3) = 4.17, P = 0.24) nor in midterm scores between sections in Fall 2015 (χ 2 (1) = 0.44, P = 0.51), the sections for which midterm scores were available.

A. Social network analysis
Since the interaction data is relational in nature, we use social network analysis to examine students' integration. Centrality is a family of measures that quantify the relative importance of individuals in a network. From a methodological point of view, centralities provide a unique empirical way to understand students' structural integration into social groups, i.e., it sheds light on student integration via their social ties.
Various social relationships bear different meaning. Thus, treating every contact in the same manner may be overly simplistic. To more fully account for the richness of real life we modify the standard measures to include the frequencies (weights) of interactions (ties) between students (nodes) in the MI courses. For directed degrees, we use the generalization of Opsahl et al. [20] that accounts for both the ties' weights (strength) and their number (degree). The proposed measure is a product of the node's directed degree and strength averaged by degree, with the parameter α tuning the relative importance of each factor. Formally, where α ∈ [0, ∞) and C ← /→ D is node's directed degree, i.e., Here, x ji (x ij ) take on value 1 if node j sends a tie to node i (node i sends a tie to node j), and 0 otherwise.
with w ji (w ij ) denoting the weight of a tie from node j to node i (from i to j), taking value 0 if a tie is not present. It is easy to see that if α = 0, then C α T = C D and if α = 1, then C α T = C S . When α ∈ (0, 1), having many weak connections is emphasized more than having only a few strong ones (for the same total strength, Eq. (3)). On the other hand, when α > 1, it is favorable to have a few strong connections (for the same total strength, Eq. (3)).
The weighted extension for closeness is where d α ij = min w −α im + · · · + w −α nj is the weighted shortest path linking i and j. When α = 0, the binary version of closeness results (i.e., the weights are ignored), while for α = 1 only the weights are important. For α < 1, a shorter path of weak ties is favored over a longer path with strong ties and for α > 1 the reverse is true (i.e., the number of intermediary nodes is less important than the strength of the ties).
To truly understand one's involvement in a network it is important to consider not only how well-connected a given person is, but also how meaningful his/hers connections are. The generalizations above give an approach that captures the nature of interactions of interest. Moreover, these measures allow for control of the relevant impact of both the number of connections, as well as their weight (frequency). It has been argued that sharing of complex knowledge -i.e., knowledge that is hard to articulate or can be acquired only through experience -requires strong ties [21], while for easily codified or explicit knowledge, weak ties are more important [22]. Thus, to emphasize the number of ties, while also taking into account their strength, we chose α = 0.5.
For eigenvector centrality, we use the standard extension to weighted graphs: for each node, we sum all frequencies of connections to others weighted by their degrees. The eigenvector centrality is given by the solution to where A is the adjacency matrix of the network, with matrix elements a ij taking on the weight value of a tie TABLE III. Comparison of network characteristics. Basic network descriptors for the in-and out-of-class networks from the third collection (week 8) for all section: network size (n), network density (∆), diameter (the length of the longest path between two nodes; D), average path length (the shortest path between two nodes, averaged over all pairs of nodes; L), transitivity (a fraction of nodes that share a neighbor "a friend of a friend is also my friend"; Tr), reciprocity (a fraction of reciprocal connections; ρ ↔ ). from node i to node j, λ max is the greatest eigenvalue of A and C E is an eigenvector of A corresponding to λ max . The eigenvector centrality of node i is given by the i th component of vector C E .

Modeling Instruction classroom network
Identifying students who are likely to not persist when there is still time for corrective action gives additional opportunities to retain students. We therefore want to examine data for the earliest possible collection during the semester that gives meaningful information. A recent study found that network data collected as early as week 8 (which is mid-semester) leads to informative measures for predicting students' academic performance [23]. Thus, in our analysis, we focus on the collection that took place at the end of week 8 in three out of the four sections and at the beginning of week 9 in the fourth. As can be seen in Fig. 2, the evolution of the out-of-class networks shows that by mid-semester all networks are basically fully developed and nearly all students (about 95 %) who will end up being in the network by the end of the semester are already present, further supporting this choice.
In our analysis, we consider four in-class networks and four-out-of-class networks (see Table III for the basic network characteristics). Centrality measures are calculated separately for each network and then the resulting indices from all sections are aggregated to represent one distinct variable (for each measure). As mentioned above, the frequencies are incorporated in the survey question for the in-class network (see Appendix S1 in the SM). For the out-of-class network, though, students were asked only to report interactions that took place, but not to rate them. To account for repeated out-of-class interactions we aggregate all "up-to-date" networks. We do so by pooling answers to the question about out-of-class interactions from the first (completed at the end of week 2), second (completed at the end of week 5 or at the beginning of week 6), and third (completed at the end of week 8 or at the beginning of week 9) survey. Interactions that took place only once during the first 8 weeks of the semester have weight 1, while reoccurring interactions have higher weights (either 2 or 3). Descriptive statistics for the four centralities are presented in Table IV. Based on the Shapiro-Wilk test we reject the null hypothesis about the normal distribution for all measures.

Handling missing data
The network data collections took place in the classroom at the end of class. For the in-class network, the enrollment in the Modeling Instruction course defines the network boundary. That is, all students enrolled are included in the final roster. Since on any given day some of the students where not present and others had to leave . Student involvement is taken as the percentage of students enrolled in the course who either listed other students or were listed by peers on the out-of-class part of the SNA survey. In three out of four sections, over 90% of students are in the network by week 8 (for the Fall 2015A section, the values are consistently lower by about 20%). Note that the third collection was administered in week 8 in three out of the four groups (i.e., Fall 2015A, Fall 2015B, Fall 2016A) and at the beginning of week 9 in one group (i.e., Fall 2016C). (B) When normalized by the size of the network at week 13, it becomes clear that the network evolution is comparable between sections and that the equilibrium is basically reached by mid-semester (week 8, the third collection). (C) An example of the out of class network at week 2 and 8 for section Fall 2015B (see Fig. S1 in SM for the other sections). At week 2, only a few connections are present. By week 8, however, a strong network of students exists, with only a handful of students not embedded within the network. The size of the nodes in both networks corresponds to closeness at week 8 and the color indicates whether a node is a student (magenta) or a person that was not enrolled in the course (gray).
the classroom before if officially ended (e.g., to go to their next class), none of the surveys had a 100 % response rate. The overall response rates, however, were comparable between sections (M = mean, SD = unbiased estimation of a standard deviation: M F15A = 84 %, SD F15A = 8 %; M F15B = 87 %, SD F15B = 6 %; M F16A = 79 %, SD F16A = 11 %; M F16C = 80 %, SD F16C = 7 %). Using the Kruskal-Wallis rank sum test, we find no significant difference for the response rates between sections (χ 2 (3) = 2.36, P = 0.50). Centralities are fairly robust to random missing data. In particular, for networks with sizes comparable to our data (i.e., 40 to 75 nodes) up to 35 % of missing data for directed degrees and about 20 % for closeness and eigenvector did not affect the overall structure of the network [24]. Thus, students who do not appear on a given collection are treated as "isolates" (disconnected members of a network) and their relevant centrality values are calculated accordingly. For the outof-class network, though, students who did not appear in the network on any of the five collections are removed from the roster (20 individuals in Fall 2015, 3 individuals in Fall 2016).

Statistical analysis
For the statistical analysis of the network data, we use the R programming language [25], and, in particular, the igraph [26] and tnet [27] packages. The Shapiro-Wilk test was used to test for normality of the centrality measures, the Kruskal-Wallis rank sum test was used to test for statistically significant differences between sections and between different grade levels, and the chi-squared test was used to test for significant differences between sections and the three grade groups in terms of gender, ethnicity and academic plan.
The statistical significance of independent variables within nested models was verified using the likelihood ratio test (LRT), with the null hypothesis stating no difference in the fit to the data (and thus the simpler model, with less variables, is better). LRT uses the ratio of the maximized value of the likelihood function for the full model over the maximized value of the likelihood function for the simpler models.
To account for the false discovery rate, the Benjamini-Hochberg procedure was implemented. We consider results with P < 0.05 as significant. All protocols in the project were approved by the Florida International University Institutional Review Board (IRB-13-0240 exempt, category 2). Details about the statistical analysis procedures are presented below.

Logistic regression analyses
The dependent variable in our study is dichotomous (persistence or lack thereof). To measure the relationship between students' centralities, background information and persistence, we used logistic regression. In the first stage, we wanted to identify centralities that carry significant information about the odds of persisting. To do so, we run 8 simple logistic regression models (four for the in-class and four for the out-of-class centralities) with a single centrality as a predictor. All measures are found to be statistically significant and thus we include all of them in further analysis. Analyses are performed with listwise deletion of missing data. In particular, students who did not report out-of-class interactions on any of the five collections were not included in the out-ofclass network and thus centralities for those individuals are not available. The odds ratio (OR) for indegree and outdegree implies how each additional interaction (i.e., increase or decrease of the respective measure by one, respectively) affects the odds of persisting. The direct effect of individual ties on closeness and eigenvector is much less intuitive and depends strongly on how the topology of the network changes with the additional ties. For example, adding a tie between two otherwise separate groups will have much larger effect on closeness than adding a tie within already a well-connected group [28]. Moreover, both measures are normalization and take on values between 0 and 1, inclusive. Thus, the odds ratio for closeness and eigenvector is thus rescaled to indicates how odds of persisting change if the measure for an individual increases by 5 %. Similarly, OR for grade is rescaled to reflect the increase by a partial letter grade (i.e., from B to B+).
The second stage of our analysis is exploratory. Due to a lack of pre-existing theory on how in-and outof-class centralities should be combined, we consider 16 models with two measures (one of each network type), as well as students' demographic information (i.e., persistence ∼ ic.centrality + oc.centrality + gender + ethnicity + major + grade). Since we make no assumptions regarding the relationships between the variables, the stepwise regression is an appropriate method for identifying best models. For each model, the Akaikes Information Criterion (AIC)-based stepwise-selection is used to identify all variables that can be removed while minimizing the estimated information loss (implemented in R via the step function). This procedure identifies models that give the best fit to the persistence data without over-fitting it (i.e., the removal of the least informative variables). When no more variables can be deleted, the exploratory stage is completed. Our hypothesis is that including an out-of-class centrality in a model should improve the predictive its power over only an in-class centrality (and, possibly, demographic information). Thus, we consider for further analysis only models with two centralities and, possibly, other factors.
Comparison of the AIC values allows us to determine the relative loss of information for non-nested models. Based on AIC values for the five models obtained via stepwise-selection we identify three additional models that can be excluded from analysis. The variance inflation factor for the remaining two models, ranging from 1.00 to 1.04, indicates no collinearity between variables.

RESULTS
To answer the leading question of our study: Do out-of-class networks -the "networks of choice" -improve models for predicting persistence within the introductory Modeling Instruction sequence?
we examine various centrality measures and other factors to determine which ones are the most important. We start by looking at models with a single centrality for the networks (see SI Materials and Methods for explanation of the various centrality measures used in the analysis). Previous studies show that centrality measures tend to be correlated, resulting in redundancy when using more than one in a model [29,30]. Thus, when considering just one network type (in-or out-of-class), we do not consider models with more than one measure. We then build more complex models that relay on two centralities (one for in-and one for the out-of-class network), as well as students' information. Our goal is to identify the simplest, accurate model of persistence.

Single predictor models
In the first stage of our study, we use simple logistic regression models for persistence as predicted by various centralities (i.e., M ic : persistence ∼ ic.centrality for inclass models; M oc : persistence ∼ oc.centrality for outof-class models, where ic.centrality and oc.centrality denote measures for the in-class and out-of-class networks, respectively). In particular, we examine indegree, outdegree (local, individual node-level measures), eigenvector (intermediate between local and global measures) and closeness (global measure). Our outcome variable is dichotomous (i.e., persistence or lack of thereof). For both in-and out-of-class networks, those instructors who are reported on the survey become a part of the network. For in-class networks, this results in including all instructional staff (i.e., instructors, TAs and LAs). This is unsurprising since they are an important source of support (be it academic or social) and interactions in class are convenient. For both networks, we want to capture the frequency of interactions, which is done as described in the Methods section.
Estimates for each measure and their significance levels are in Table V. When examining correlations between only persistence and one centrality, all centralities are significant predictors for both in-and out-of-class networks, which is in-line with a prior study on in-class networks only [7]. Thus, we will consider all measures in order to find the most informative.

The power of grade
In a recent study, Zwolak et al. found that the final grade is the only factor that improves the predictive power of models with an in-class centrality [7]. A simple logistic regression with grade as a sole predictor for persistence (M gn : persistence ∼ grade) confirms a positive correlation between final grade and persistence for our data (odd ratio OR = 1.5; estimate B = 1.36, standard error of estimate SE = 0.19, significance level P < 0.001; sample size N = 273). It seems natural that students who score well in class are likely to continue, while students who either fail the class or score poorly will not (and, in cases of failure, they are prohibited from continuing). The behavior of students "in the middle of the pack" is less obvious. Thus, to gain a better understanding for how final grades affect student's decision whether to persist, we split the data into three groups based on grades. An inspection of final grades leads us to identifying as cutoff points grades B+ and C. This division gives us three fairly equitable groups: "high" (grades A, A-, B+), accounting for 34% of grades, "intermediate" (grades B, B-, C+), including 37% of grades, and "low" (C and lower), including 29% of grades. We found no difference between the groups in terms of gender (chi-squared test: χ 2 (3) = 0.63, P = 0.89), ethnicity (chi-squared test: χ 2 (3) = 2.73, P = 0.43), and academic plan (chisquared test: χ 2 (9) = 14.7, P = 0.10). In what follows we focus on the intermediate category, though we will verify applicability of the identified models also for all grade levels.

Analysis of the full models
Since different centrality measures capture different aspects of networks, and the networks themselves for inand out-of-class interactions are different in nature (i.e., interactions of convenience versus interactions of choice), it is not obvious how to combine these indices. Thus, in the final step, we examine various combinations of the significant in-and out-of-class centralities. Again, to avoid redundancy, we use only one measure for each network type, i.e., one in-class and one out-of-class, in each model [29,30]. This gives us a total of 16 models with two measures, as well as students' demographic information (i.e., M F : persistence ∼ ic.centrality + oc.centrality + gender + ethnicity + major + grade).
For each model, we employ stepwise-selection based on Akaike's Information Criterion (AIC) to identify all variables that can be removed while minimizing the es- timated information loss. This procedure identifies 5 candidate models with three predictors (an in-class centrality, an out-of-class centrality and the final grade) that best fit to the persistence data without over-fitting it. Further comparison of the relative quality of these 5 models (as measured by the AIC) allows us to rank them in terms of likelihood of minimizing information loss. In particular, a model that includes in-class outdegree and out-of-class closeness (i.e., M oc : persistence ∼ ic.outdegree + oc.closeness + grade) is found to give the best fit to the data. The second best model relies on in-class closeness and out-of-class outdegree (i.e., M co : persistence ∼ ic.closeness + oc.outdegree + grade) and is 0.77 times as probable as the first model to minimize the information loss. The remaining three models perform about 3.2 times worse than the best model and thus we exclude them from further analysis. Consequently, this leaves two candidate models for predicting persistence among students "in the middle of the pack". Interestingly, both models include a combination of outdegree and closeness, the only two measures that for the in-class network alone improve the predictive power of grade [7].
In the next step, we employ the likelihood ratio test to further investigate the two best models (i.e., M co and M oc ) for the effect of removing additional variables (one at a time) on the overall fit to the data. We find that, for the intermediate grade cohort, only the out-ofclass centrality carries statistically significant information about persistence. Removal of grade or in-class centrality does not affect the fit of either model (see Ta- In other words, when all grades are considered, the out-of-class closeness or outdegree combined with final grade gives the best prediction of persistence. These are surprising findings that we will discuss momentarily.

Model verification
We determined that, for students with final grades in the intermediate range (i.e., B, B-and C+ grades), the best models of persistence are single-predictor models with either out-of-class outdegree or out-of-class closeness (i.e., M outdeg : persistence ∼ oc.outdegree and M close : persistence ∼ oc.closeness, respectively). Applying these models to the intermediate final grades confirms that both out-of-class measures are statistically significantly correlated with persistence (see Table VII).
Identifying students who are less likely to persist is important when there is still time to take actions to help them. Thus, using the final grade makes the models illsuited for practical purposes. Since midterm grades are traditionally used to identify students who are at risk of failing the class (and may also be used to predict persistence), we test our models using midterm scores as a proxy for final grade. While there are three tests throughout the semester in the MI courses, the first midterm occurs around the same time as the third collection. Thus, we use scores from this test to approximate final grades.
As expected, we find a positive correlation between the midterm scores and final grades (generalized linear model: residual dev. = 104.14, df = 1, P < 0.001; N = 143) and midterm scores and persistence (logistic regression: OR = 1.03; B = 0.10, SE = 0.02, P < 0.001; N = 143). Moreover, when dividing midterm scores into three categories introduced earlier, we find a positive correlation also between the midterm scores and fi-TABLE VII. Model verification without grade. Odds ratio (OR), estimates and standard error (SE) for simple logistic regression models for persistence as predicted by out-of-class outdegree (M outdeg ; columns 3-5) and out-of-class closeness (M close ; columns 6-8). We consider 3 cases to test our modes: When all grades are considered, we found that models with a centrality for out-of-class network and a grade give the best fit to the data (i.e., M og : persistence ∼ oc.outdegree + grade and M cg : persistence ∼ oc.closeness + grade). We test these two models in a similar manner as we did for the intermediate cohort.
We consider two cases: first we use final grades as a predictor, then we take midterm scores as a proxy for final grades. We find that in both cases the centrality and grade are significant predictors for persistence (see Table VIII).
Finally, to make practical use out of these results, one would like to know, e.g., the value of out-of-class closeness for which students may be at risk of not persisting. When considering grade, it is clear that having a C or less puts one at risk. However, where such a threshold should be drawn (C, C+, B-, etc.), it is not clear since the intermediate grade levels (C+, B-, and B) are not significantly correlated with persistence. For closeness, one can find the threshold value for the intermediate cohort. It comes at about 0.14 (the significance of this split is confirmed by the chi-squared test: χ 2 (1) = 7.46, P = 0.006). Below this value, students have about a 63% chance of persisting. Above this value, students have about a 92% TABLE VIII. Model verification with grade. Odds ratio (OR), estimates and standard error (SE) for simple logistic regression models for persistence as predicted by out-ofclass outdegree and grade (Mog; columns 2-4) and out-of-class closeness and grade (Mcg; columns 5-7). We consider two test cases for the grade variable: (1) final grades (N = 239); (2) midterm scores (N = 144). The odds ratio for grade is rescaled to reflect the increase by a partial letter grade (i.e., from B to B+). Significant p-values are marked with an asterisk.

Coefficient
Mog chance of persisting. This large difference is a reflection of the importance of closeness.

DISCUSSION
Internal communities, such as learning groups within and out of a classroom, are among the most important factors indicating one's integration into the social structure of a university. Previous studies on the effect of integration on persistence found that outdegree and Bonacich's power (a generalization of eigenvector centrality) had a direct, positive effect on students' persistence [5]. This work, though, did not separate the in-and out-of-class networks, a fine-graining of the network data that we introduce and find above to be highly informative. Moreover, while these previous findings might guide the development of activities (academic and social) intended to encourage the formation of students' networks "designed to enhance. . . diversity and foster opportunities for nurturing and connecting emerging students social leaders", the data used in the analysis was gathered once the semester was over, precluding any intervention intended to help at-risk individuals.
In order to successfully promote persistence at the individual level, be it through structured and purposeful mixing of students in class or handing out group assignments for out-of-class practice, one has to not only identify reliable measures that can indicate students at risk, but also do it in a timely manner, when the semester is still in progress. In our study, we look at the network data at mid-semester, when there is still plenty of time to intervene. Additionally, we adjust the network methodology to fully utilize the richness of real interactions, which are weighted in nature.
Although it has been noted that the effect of social and academic integration "will vary. . . inside and outside the classroom" [10], previous studies did not explicitly distinguish between those two types of interactions [5,6].
In an attempt to better understand integration within various types of networks, we explicitly ask students to report both in-and out-of-class interactions. Since current technological advancement diminishes the spatial and temporal limitations for communication, we ask also for reports of the non-face-to-face interactions. Finally, we examine a minority-serving institution, where most students (about 92%) live off-campus [31], and many are from low income backgrounds (approximately 50% receive Pell grants [32]. For all these reasons, our case expands and, in important ways, complements previous studies that looked at "upper middle-class, full-time, residential students" [5]. The leading question for our analysis was whether inclusion of the out-of-class interactions improves the proposed model of persistence [7]. To test this hypothesis, we build models with various combinations of in-and outof-class centralities combined with students' demographic information (i.e., gender, ethnicity, academic plan, final grade). Consistently with previous studies [33], we find lack of the effect of gender, ethnicity, etc. on the fit to the data.
Position within the out-of-class "network of choice", however, not only improves the model of persistence but is actually more informative than the in-class "network of convenience" and, in some cases, grades. In particular, midterm grades usually serve as a proxy for final grades and have traditionally been used to identify students at risk of failing the course. For students performing very well (and therefore likely to persist) and for students who perform rather poorly (and thus are either more likely to drop or -in case of failing grade -cannot continue), the final grade alone is the best indicator of the likelihood of persistence. This remains true when performance throughout the semester is used to approximate final grades.
For students with average to good performance (i.e., C+, B-, and B) -students that you want to keep but who traditionally are not considered to be at risk of dropping out -the grade alone does not provide sufficient information to determine the odds of persistence. On the contrary, we find no correlation between grades "in the middle of the pack" and persistence, whether we use final grades or their midterm proxy. Moreover, although in-class centralities were positively correlated with persistence when looked at separately, and were expected to carry the most significance aside from final grade, we found that in comparison with out-of-class centralities they lost significance. Instead, the level of connectedness (as measured by outdegree) and the overall embeddedness within the network (as measured by closeness) for the network of choice are significant predictors for persistence. In other words, students who are most likely to persist are those who are well-immersed into the out-ofclass social system of the university and are successfully reaching out to others.
More quantitatively, when a student's closeness is below about 0.14, then the chances of persisting are about 63%, but they are about 92% above that value of closeness. Thus, when grades alone do not provide such information, the awareness of a student's out-of-class social and academic integration is the best indicator of the likelihood of persistence. This is especially important for students whose grades place them "in the middle of the pack" and who do not otherwise seem at risk of not persisting. For those individuals, their position within the network of choice, established as early as mid-semester, can be successfully employed to identify students who, despite satisfactory grades, actually need help or encouragement. Providing them with opportunities to build a strong out-of-class network of support may increase their odds of persisting.
One obvious question is why does the in-class network pale in comparison to the out-of-class network? A potential clue comes from the active engagement approach of the MI classroom. When interactions are strongly encouraged -and, indeed, required in some cases -this will tend to "squeeze" the number of ties/interactions into a narrow range: Everyone interacts a lot. Even still, inclass network measures still retain significance on their own. They only lose significance when compared to outof-class measures. This suggests that it is students who really become part of the social fabric surrounding the university -not just present but developing ties to others -are the ones that become, or are, committed to their education.

Implications
Building a network of academic or social support is more difficult in commuter and very large schools. Participation in an active-learning course enables students to develop a network of supportive peers that might help them make an easier transition from high school to college and also to integrate them into the community. MI provides an environment where students can learn and build a network at the same time. This is particularly valuable for nonresidential students.
MI is an example of a curriculum that strongly emphasizes the importance of collaborative learning and, in the process, promotes the culture of working together. From the network of choice evolution, shown in Fig. 2, one can see that by the third collection -about half way through the semester -the out-of-class networks are basically fully developed. The great majority of students report working together out-of-class at least once.
In large introductory courses, with hundreds of students enrolled in each section, it is much harder to create a collaborative environment [18]. When practicing active-engagement pedagogy is more difficult, promoting out-of-class collaborations can be especially beneficial for students. The significance of outdegree and closeness (but not indegree) for out-of-class models suggests that students' participation in the network of choice provides them not only with the academic support, but also the support needed to balance the struggles faced in class (or in personal life). In turn, access to such support might influence students' decision to continue in college, despite struggles and challenges. Interviews with students further support this hypothesis [cite LE PERC].
Our findings shed light on one of pressing questions of our time: How to increase persistence in STEM education? Modern technologies allow for fast electronic collection and analysis of network data. Knowing when is the best time for data gathering allows one to conduct the collection only once. Network surveys can be taken by students at the end of a lecture, on their phones or laptops. Then, the data can be processed electronically to obtain the metrics of interest, placing no additional burden on instructors. Regardless of whether a passive (e.g., designing the course/coursework) or active (e.g., encouraging individual students) approach is taken, the results presented above suggest that promoting of outof-class interactions is a simple method to increase persistence (and, ultimately, the number of STEM degrees awarded).

Limitations and next steps
Although the data for our analysis was collected in an introductory physics course, students enrolled in the course spanned over 20 different majors across the uni-versity. Thus, they constitute a representative sample of students enrolled in STEM majors. With that being said, the clear next step is to extend this approach to different areas of study and to more traditional curricula. This will help to determine the utility of SNA for increasing and understanding students' persistence more broadly. Importantly, determining whether the network of choice (opposed to the network of convenience) is the most significant factor will be helpful in employing these findings to increase persistence.
In addition to major and pedagogical approach, institutional properties may also be important. We observed that the development of the out-of-class network takes about 8 weeks in the semester system. Thus, it is important to examine different academic calendar systems, such as quarters and trimesters, as there is less time for the network to fully evolve. As well, since FIU is a Hispanic-Serving Institutions, our study includes mostly Hispanic and Latino(a) student population. While this complements previous studies on predominantly non-minority populations, all groups should be considered in future work.
Student ID:_________________________ Name:_________________________ 3/3 Question 1: Please choose from the presented list people from your physics class that you had a meaningful interaction with in class this week, even if you were not the main person speaking or contributing. You may include names of students outside of the group you usually work with. You don't have to fill in all columns. You may use the name of a student or their corresponding number.
I had a meaningful interaction with these people this week in class... … one time. … more than one time but NOT every day. … every day.
Question 2: Throughout this week, did you work with anyone on physics-related material outside of class, either in person or virtually (using, e.g., WhatsApp, Google Chat, etc.)? If so, whom did you work with? (Please provide their FIRST and LAST name. If you worked with someone not from your class, provide their major if possible.) If not, please write NONE. If you worked with someone from a school-related group you participate in, please name that group. Also, please name instructors or TAs if you met with one or more out-of-class. Fall 2016C (C). In each case, the number of connections at week 2 (left) is much lower than at week 8 (right). The size of the nodes in both networks corresponds to closeness at week 8 and the color indicates whether a node is a student (magenta) or a person that was not enrolled in the course (grey).