Applying a mathematical sense-making framework to student work and its potential for curriculum design

This paper extends prior work establishing an operationalized framework of mathematical sense making (MSM) in physics. The framework differentiates between the object being understood (either physical or mathematical) and various tools (physical or mathematical) used to mediate the sense-making process. This results in four modes of MSM that can be coordinated and linked in various ways. Here, the framework is applied to novel modalities of student written work (both short answer and multiple choice). In detailed studies of student reasoning about the photoelectric effect, we associate these MSM modes with particular multiple choice answers, and substantiate this association by linking both the MSM modes and multiple choice answers with finer-grained reasoning elements that students use in solving a specific problem. Through the multiple associations between MSM mode, distributions of reasoning elements, and multiple-choice answers, we confirm the applicability of this framework to analyzing these sparser modalities of student work and its utility for analyzing larger-scale ($N>100$) datasets. The association between individual reasoning elements and both MSM modes and MC answers suggest that it is possible to cue particular modes of student reasoning and answer selection. Such findings suggest potential for this framework to be applicable to the analysis and design of curriculum.

Figure 1

A representation of mediated cognition, showing the mediated (by a tool) and direct pathways by which a subject interacts with an object. Drawn from Vygotsky [9].

Figure 2

Left: the four modes of the MSM framework, defined by the tool and the object of sense making. Right: a sense-making diagram, showing the processes of translation and coordination that lead to a coordinated reasoning structure.

Figure 3

A representation of the finite, double square well, and the first two energy eigenstates (wave functions).

Figure 4

A schematic of the photoelectric effect experiment, taken from the PhET simulation [29]. Light incident on the metal plate ejects electrons which travel to the right. The rate at which electrons reach the right plate is measured by the ammeter, the max current is a measure of the ejection rate and is independent of the battery voltage when the voltage is positive. If the battery is turned around (the voltage is negative), the electrons are slowed down and the less energetic electrons no longer cross the gap. The voltage at which no electrons make it to the right plate is called the stopping potential ($V_0$) and is a measure of the kinetic energy of the most energetic electrons (${\mathrm{KE}}_{\max }$).

Figure 5

Left: Plots of kinetic energy versus frequency as defined by Eq. (1) (dashed and plotted in red) and Eq. (2) (solid and plotted in blue, the canonical representation), which differ in their implicit [Eq. (1)] and explicit [Eq. (2)] inclusion of the cutoff frequency. Right: A plot of the max kinetic energy of the ejected electrons versus the wavelength of light as described by Eq. (3). Note that this is a noncanonical equivalent representation to KE vs $f$ and that the minimum cutoff frequency has become a maximum cutoff wavelength.

Figure 6

Version A of the KE vs $\lambda$ task is framed in a mathematical context and responses are multiple choice. This question was asked in both semester 1 and semester 2 on the pretest (week 1) and again on the second homework (week 3). The homework included a free response follow up asking students to link this abstract mathematical formalism to the photoelectric effect experiment.

Figure 7

Version B of the KE vs $\lambda$ task is framed in a physical context and student responses are free response including a sketch of the graph. This question was asked on the exam (week 5) in semester 1.

Figure 8

Version C of the KE vs $\lambda$ task is framed in a physical context and student responses are multiple choice. This question was asked on the exam (week 5) during semester 2. Note that the labels [(a) and (d)–(g)] are used here to consistently indicate the associated plots throughout this paper; however, they were not labeled as such on the exam.

Figure 9

Left: An example of a plot coded as other that bears a strong resemblance to one of the lettered plots. The two circles are marks made by the grader. Right: Examples of the novel plots (g) and (h), emergent from the version B free response exam data.

Figure 10

Example student reasoning and associated MSM reasoning structure for plots (d)–(g). The color of the background (green or yellow) holds no meaning, these were simply the colors of the paper used for the two versions of the tests. The test versions were identical save for the ordering of some earlier questions. Note that the circle drawn on the tail of the graph in plot (d) was drawn by a grader, not the student.

Figure 11

The distribution of reasoning elements used by students who drew plots (d)–(h) on the semester 1 exam (version B of the KE vs $\lambda$ task). The colors refer to our categorization of these tools as dominantly mathematical (red), physical (blue), coordinated (green), or indeterminate (gray). As inverse (s) is automatically included in full equation, the second bar is always greater than or equal to the first. The null hypothesis, that the distribution of reasoning elements is independent of plot, is rejected at the $p\ll 0.05$ level when comparing all five plots and the $p<0.05$ level when comparing only plots (e)–(g).

Figure 12

The distribution of plots drawn based on the use of a given reasoning element. The horizontal lines show the percentage of overall responses corresponding to each graph, e.g., 39% of students drew plot (f)—the correct answer. These plots do not necessarily add up to 100%, as plots (a), (b), (h), and other are excluded from this figure. The overall percentage of student responses (regardless of plot drawn) that included each reasoning element is shown below each distribution, e.g., full equation was present in $57/114=50\%$ of students responses.

Figure 13

Results for version C on the semester 2 exam (blue) and version B on the semester 1 exam (gray)—data for semester 1 are shown in Table 2. The plot labels have been updated for consistency with the labels used throughout this paper.

Figure 14

Responses to the three implementations of the KE vs $\lambda$ task for semester 1. Both the pretest ($N=108$) and homework ($N=109$) implementations were version A—though the homework included a free response, physics-context follow-up—and the exam ($N=114$) was version B. Plots (a)–(f) correspond to the plots from version A shown in Fig. 6 and plots (g) and (h) (emergent from the free-response exam) are shown in the right of Fig. 9. Responses coded as o (other) make up less than 10% of the overall exam responses.