Assessing expertise in introductory physics using categorization task

The ability to categorize problems based upon underlying principles, rather than surface features or contexts, is considered one of several proxy predictors of expertise in problem solving. With inspiration from the classic study by Chi, Feltovich, and Glaser, we assess the distribution of expertise among introductory physics students by asking three introductory physics classes, each with more than a hundred students, to categorize mechanics problems based upon similarity of solution. We compare their categorization with those of physics graduate students and faculty members. To evaluate the effect of problem context on students’ ability to categorize, two sets of problems were developed for categorization. Some problems in one set included those available from the prior study by Chi et al. We find a large overlap between calculus-based introductory students and graduate students with regard to their categorizations that were assessed as “good.” Our findings, which contrast with those of Chi et al., suggest that there is a wide distribution of expertise in mechanics among introductory and graduate students. Although the categorization task is conceptual, introductory students in the calculus-based course performed better than those in the algebra-based course. Qualitative trends in categorization of problems are similar between the non-Chi problems and problems available from the Chi study used in our study although the Chi problems used are more difficult on average.

Figure 1

Histogram of algebra-based introductory physics students (total 109 students) who categorized various percentages of the 25 problems in version II of the problem set in “good” categories when asked to categorize them based on similarity of solution. The 7 Chi problems were categorized worse than the other 18 problems showing that the nature of introductory physics questions is important in students’ ability to categorize them. The percentages of students for all 25 problems taken together are also shown. The error bars in all graphs show the standard error.

Figure 2

Histogram of the algebra-based introductory physics students (for both versions I and II of the problem set) who categorized various percentages of the 25 problems in “good” categories when asked to categorize them based on similarity of solution. Version II involving Chi problems was categorized worse than version I. As discussed in the text, there is no statistically significant difference between the performance of the two algebra-based classes on the 15 problems that were common to the two versions. The error bars refer to the standard error.

Figure 3

Histogram of calculus-based introductory physics students, graduate students, and physics faculty who categorized various percentages of the 25 problems in version I in “good” categories when asked to categorize them based on similarity of solution. Physics faculty members performed best in the categorization task followed by graduate students and then introductory physics students, but there is a large overlap between the graduate students and the introductory physics students. Reprinted with permission from C. Singh, Am. J. Phys. 77, 73 (2009). Copyright 2009, American Association of Physics Teachers.

Figure 4

Histogram of the calculus-based and algebra-based introductory physics students (version I of the problem set) who categorized various percentages of the 25 problems in “good” categories when asked to categorize them based on similarity of solution. The calculus-based introductory students categorized the problems better than the algebra-based introductory physics students. The error bars refer to the standard error.