Physics Inventory of Quantitative Literacy: A tool for assessing mathematical reasoning in introductory physics

One desired outcome of introductory physics instruction is that students will develop facility with reasoning quantitatively about physical phenomena. Little research has been done regarding how students develop the algebraic concepts and skills involved in reasoning productively about physics quantities, which is different from either understanding of physics concepts or problem-solving abilities. We introduce the Physics Inventory of Quantitative Literacy (PIQL) as a tool for measuring Quantitative Literacy, a foundation of mathematical reasoning, in the context of introductory physics. We present the development of the PIQL and evidence of its validity for use in calculus-based introductory physics courses. Unlike concept inventories, the PIQL is a reasoning inventory, and can be used to assess reasoning over the span of students’ instruction in introductory physics. Although mathematical reasoning associated with the PIQL is taught in prior mathematics courses, pretest and post-test scores reveal that this reasoning is not readily used by most students in physics, nor does it develop as part of physics instruction—even in courses that use high-quality, research-based curricular materials. As has been the case with many inventories in physics education, we expect use of the PIQL to support the development of instructional strategies and materials—in this case, designed to meet the course objective that all students become quantitatively literate in introductory physics.

Figure 1

Examples of symbolic forms represented in the PIQL. (a) Creating Quantity: Measurement [19] and Quantity [20], and (b) Covariation: ratio, dependence, scaling, and opposition [11].

Figure 2

Workflow for developing and revising items, piloting individual items using interviews, assembling versions of the PIQL, and collecting data with the instrument as a whole. We consider the most recent version of the PIQL to be stable because qualitative and quantitative data analyses provide strong evidence for its validity as a tool for measuring students’ PQL (see Sec. 4).

Figure 3

PIQL MCMR item that exemplifies a proceptual understanding of the negative sign in a mathematics and physics blend. The correct responses are D and G.

Figure 4

The initial development of items for the item library followed the processes shown above. The entire process applies to items that were developed from scratch. Items with existing evidence of validity, or those that had only a small finite number of possible answer options, enter the process at the top of the right side as indicated above.

Figure 5

Current version of the Charged Spheres item on the steady-state PIQL, which has withstood multiple challenges to its validity, intended to assess student reasoning about the negative sign in the context of electric charge. Answer B is correct.

Figure 6

CTT difficulty (a) and discrimination (b) parameter distributions for all versions of the PIQL. Successive versions are ordered from bottom to top. The desired range of difficulty values is between 0.2 and 0.8 (shown by dashed red lines). The desired range for discrimination is above 0.3.

Figure 7

Distribution of PIQL scores for each course in which data were collected. Data include 2758 response sets from 2078 individuals who responded to PIQL v2.2 in four different academic terms. (Some students provided data in multiple courses at various times.)

Figure 8

Fraction of student responses in each category of our four-level scoring scheme for MCMR items. Items 17 and 18 have multiple correct answers. These results are from PIQL v2.2.

Figure 9

Item response curves for three items on PIQL v2.2. Each plot shows the fraction of students who chose each response out of the students who earned each score on the total test. Item 14 has correct answer B, item 17 has correct answers A, C, and D, and item 18 has correct answers D and G.