Visualization of longitudinal student data

We use visualization to find patterns in educational data. We represent student scores from high-stakes exams as flow vectors in fluids, define two types of streamlines and trajectories, and show that differences between streamlines and trajectories are due to regression to the mean. This issue is significant because it determines how quickly changes in long-term educational patterns can be deduced from score changes in consecutive years. To illustrate our methods, we examine a policy change in Texas that put increased pressure on public school students to pass several exams, and gave them resources to accomplish it. The response to this policy is evident from the changes in trajectories, although previous evaluation had concluded the program was ineffective. We pose the question of whether increased expenditure on education should be expected to correspond to improved student scores, or whether it should correspond to an increased rate of improvement in student scores.

Figure 1

Average score changes of eighth-grade Texas students on Texas’ high-stakes mathematics exam (TAKS), as a function of score in eighth grade, and as a function of student economic status measured by eligibility for free or reduced lunch. Average score changes across the whole state are treated as independent random events and used to construct standard error bars. That is, using data from 2003 to 2010, there are seven independent measurements of student score changes. The point to note here is that controlling for prior year score with a single straight line would be a technical error, yet some statistical models of student performance do just that.

Figure 2

(a) Example of a snapshot flow plot computed from statewide mathematics scores on the TAKS (see Sec. 4), showing average score changes of White students from spring 2009 to spring 2010. The area of each arrow is proportional to the number of students. (b) Same plot but for Hispanic students. Lines indicate commended and panel recommendation score cutoffs.

Figure 3

(a) Example of a cohort flow plot, again from TAKS mathematics scores, showing average annual score changes of White students in the class of 2012. Lines indicate commended and panel recommendation score cutoffs. (b) Same plot, but for Hispanic students.

Figure 4

These streamlines were obtained from snapshot flow plots similar to those in Fig. 2 using TAKS mathematics scores in 2005 and 2006 for low-income and well-off children, as determined by eligibility for free and reduced lunch. We disaggregate by poverty here rather than by race to illustrate the variety of comparisons that are possible. The comparisons seem rather extreme. By 11th grade, low-income students who started in the 90th score percentile at third grade are below well-off students who started at the 60th score percentile at third grade. We will show later in this article that this type of plot exaggerates differences between groups and that real trajectories do not diverge as much.

Figure 5

The flow pattern of Texas students changed with the class of 2012. We display a series of cohort flow plots. Each arrow indicates the average one-year change in mathematics scores of students with the same starting score. The size of each arrow is proportional to the number of students. Upper and lower lines indicate commended and panel recommendation scores, respectively. Students eligible for free and reduced lunch (low-income students) depicted only; well-off students showed similar improvement.

Figure 6

Streamlines computed by averaging over the data from the classes of 2005 through 2011, and from the classes of 2012 through 2016, in the cohort flow plots of Fig. 5. The thickness of the streamlines is proportional to the number of students.

Figure 7

Trajectories computed by tracing explicitly the average scores over time of cohorts of fourth graders grouped by their fourth-grade mathematics scores, before and after the introduction of SSI (ARI/AMI). Comparison with Fig. 6 shows that following cohorts of students in time explicitly rather than integrating up year-to-year changes produces trajectories that compress less towards the mean.

Figure 8

Distribution of student scores for the two cohorts before and after ARI/AMI. The distributions are shown for fourth and 10th grades. Note that the number of low-scoring students drops when moving from the first cohort to the second, while the number of high-scoring students increases. This trend appears for both better-off and low-income students, and it persists for both fourth grade and six years later in 10th grade.

Figure 9

Low-performing low-income students saw gains in all grades. Standard error bars follow from averaging data over several cohorts. For before ARI/AMI, all data are employed. For after ARI/AMI, all cohorts graduating in 2012 and after are used. However, for students in eighth grade and below, 2010 data are excluded since funding for the program dropped by a factor of 3 in 2009–2010. Data for students above eighth grade are retained on the grounds that they received the boost ARI/AMI provided, and it is reasonable to see if they retain momentum or not. The data in ninth grade, which include a contribution from 2010 when funding dropped, show a strong decrease.