Quantifying critical thinking: Development and validation of the physics lab inventory of critical thinking

Introductory physics lab instruction is undergoing a transformation, with increasing emphasis on developing experimentation and critical thinking skills. These changes present a need for standardized assessment instruments to determine the degree to which students develop these skills through instructional labs. In this article, we present the development and validation of the physics lab inventory of critical thinking (PLIC). We define critical thinking as the ability to use data and evidence to decide what to trust and what to do. The PLIC is a 10-question, closed-response assessment that probes student critical thinking skills in the context of physics experimentation. Using interviews and data from 5584 students at 29 institutions, we demonstrate, through qualitative and quantitative means, the validity and reliability of the instrument at measuring student critical thinking skills. This establishes a valuable new assessment instrument for instructional labs.

Figure 1

Distributions of respondents’ matched pre- and postscores on the PLIC ($N=1911$). The average score obtained by experts is marked with a black vertical line.

Figure 2

Average score (representing question difficulty) for each of the 10 PLIC questions. The average expected score that would be obtained through random guessing is marked with a black square. Scores on the first seven questions are statistically different between the pre- and postsurveys (Mann-Whitney U, Holm-Bonferroni corrected $\alpha <0.05$).

Figure 3

Boxplots comparing presurvey scores for students in FY and BFY labs, and experts. Horizontal lines indicate the lower and upper quartiles and the median score. Scores lying outside $1.5\times \mathrm{IQR}$ (inter-quartile range) are indicated as outliers.

Figure 4

Boxplots of students’ total scores on the PLIC grouped by the type of lab students participated in. Horizontal lines indicate the lower and upper quartiles and the median score. Scores lying outside $1.5\times \mathrm{IQR}$ are labeled as outliers.

Figure 5

Excerpt from page four of the PLIC describing the methods of the two hypothetical groups of students. The final multiple choice questions ask respondents which group they think did a better job of testing the model, while the multiple response question asks students to elaborate on their reasoning.