Assessing mathematical sensemaking in physics through calculation-concept crossover

Professional problem-solving practice in physics and engineering relies on mathematical sense making—reasoning that leverages coherence between formal mathematics and conceptual understanding. A key question for physics education is how well current instructional approaches develop students’ mathematical sense making. We introduce an assessment paradigm that operationalizes a typically unmeasured dimension of mathematical sense making: use of calculations on qualitative problems and use of conceptual arguments on quantitative problems. Three assessment items embodying this calculation-concept crossover assessment paradigm illustrate how mathematical sense making can positively benefit students’ problem solving, leading to more efficient, insightful, and accurate solutions. These three assessment items were used to evaluate the efficacy of an instructional approach focused on developing students’ mathematical sense making skills. In a quasi-experimental study, three parallel lecture sections of first-semester, introductory physics were compared: two mathematical sense making sections, one with an experienced instructor and one with a novice instructor, and a traditionally taught section, as a control group. Compared to the control group, mathematical sense making groups used calculation-concept crossover approaches more often and gave more correct answers on the crossover assessment items, but they did not give more correct answers to associated standard problems. In addition, although students’ postcourse epistemological views on problem-solving coherence were associated with their crossover use, they did not fully account for the differences in crossover approach use between the mathematical sense making and control groups. These results demonstrate a new assessment paradigm for detecting a typically unmeasured dimension of mathematical sense making and provide evidence that a targeted instructional approach can enhance engagement with mathematical sense making in introductory physics.

Figure 1

Though typical assessment approaches link calculations to quantitative questions and conceptual arguments to qualitative questions, a paradigm for assessing mathematical sense making can search for calculation-concept crossover.

Figure 2

A typical, qualitative circuit question (from Engelhardt and Beichner [43]).

Figure 3

Percentage of calculation-concept crossover approaches used in the three instructional groups on the crossover assessments. Error bars represent $\pm 1$ SEM, calculated from the binomial distribution.

Figure 4

Percentage of correct solutions of the three instructional groups on the three crossover assessments and the associated standard questions. Error bars represent $\pm 1$ SEM, calculated from the binomial distribution for binary measures.

Figure 5

Approaches used on the qualitative judgment question (exploding blocks)—calculation, conceptual argument, and both calculation and conceptual argument—along with the percentage of correct answers produced by each approach.

Figure 6

Average MS epistemology score (% of favorable responses) for each instructional group. Error bars represent $\pm 1$ standard error of the mean.

Figure 7

Scatterplot of number of crossover approaches used vs MS epistemology score, with the model fit plotted for each instructional group.

Figure 8

Two courses of expertise plotted on a 2D space of innovation vs efficiency (adapted from Schwartz, Bransford, and Sears [69]).

Figure 9

Plot of the study results by innovation vs efficiency. The dotted line represents a balance of equal levels of efficiency and innovation. The MS student average is closest to this line whereas the CTRL student average is furthest from this line.