Identifying features predictive of faculty integrating computation into physics courses

Computation is a central aspect of 21st century physics practice; it is used to model complicated systems, to simulate impossible experiments, and to analyze mountains of data. Physics departments and their faculty are increasingly recognizing the importance of teaching computation to their students. We recently completed a national survey of faculty in physics departments to understand the state of computational instruction and the factors that underlie that instruction. The data collected from the faculty responding to the survey included a variety of scales, binary questions, and numerical responses. We then used random forest, a supervised learning technique, to explore the factors that are most predictive of whether a faculty member decides to include computation in their physics courses. We find that experience using computation with students in their research, or lack thereof and various personal beliefs to be most predictive of a faculty member having experience teaching computation. Interestingly, we find demographic and departmental factors to be less useful factors in our model. The results of this study inform future efforts to promote greater integration of computation into the physics curriculum as well as comment on the current state of computational instruction across the United States.

Figure 1

Representative ROC curve for one of the 30 trials of predicting whether faculty do or do not have experience teaching computation from 44 predictor variables. The AUC for this trial is 0.8250, suggesting a good model, as the AUC is greater than 0.7.

Figure 2

Variable importances for each of the 44 factors used to predict whether a faculty member has experience teaching computation. The importance is based on the average change in the AUC if the factor is permuted. Thus, factors that change the AUC the most have the largest importances. The first five factors in color are the ones selected from the recursive backward elimination approach. The error bars represent a standard error of the mean AUC importance. Full questions can be found in Table 2 of the Appendix.

Figure 3

Distribution of responses based on whether faculty do or do not have experience teaching computation. Here, within group percentage means the percentage of faculty within the group who use computation or the group who does not use computation. Since these five factors are the meaningful factors, we expect that the distribution of responses should be different between faculty with experience teaching computation and those who do not. Plot A shows the distribution of the most important feature while plot E shows the fifth most important feature. All five plots have ${\chi }^2$ with corrected $p<0.05$.

Figure 4

Distribution of responses based on whether faculty do or do not have experience teaching computation. Here, within group percentage means the percentage of faculty within the group who use computation or the group who does not use computation. Since these two variables are less useful for predictions, we expect that the distribution of responses should not be different between faculty with experience teaching computation and those who do not.

Figure 5

Average accuracy of the model for various training fractions and number of trees in the forest. Error bars correspond to a standard error.

Figure 6

Average area under the curve of the model for various training fractions and number of trees in the forest. Error bars correspond to a standard error.

Figure 7

Fraction of the other 29 models in which the variables are selected as meaningful. Variables above the line were selected as meaningful in the original model.