Rubric for assessing thinking skills in free-response exam problems

We designed a rubric to assess free-response exam problems in order to compare thinking skills evidenced in exams in classes taught by different pedagogies. The rubric was designed based on Bloom’s taxonomy and then used to code exam problems. We have analyzed historical and recent exam problems in both algebra-based and calculus-based exams. In particular, we have examined cases where the same problem was administered on exams taught by different pedagogies. In some cases, we were able to compare different sections of the algebra-based physics course taught by the same instructor, sometimes during the same semester, one with inquiry-based instruction and the other in a more traditional lecture environment. We discuss the development of the instrument and present results. The inquiry-based students consistently demonstrated use of the thinking skills coded at least as often as the traditional students, and usually more often.

Figure 1

The rubric.

Figure 2

A problem administered to two algebra-based physics sections, one INQ, one traditional, both taught by the same instructor, the same semester, different pedagogies.

Figure 3

Examples of students answers for part (a) of the sample problem.

Figure 4

Traditional problem used in adjusting rubric.

Figure 5

Research-inspired problem used in adjusting rubric.

Figure 6

A problem administered to four algebra-based physics sections, one small (24 students) INQ in 2010, one traditional interactive engagement (80 students) in 2010, two larger (60 students) INQ sections in 2017 and 2019. All classes had the same instructor.

Figure 7

The results of the general rubric applied to two algebra-based physics sections, one INQ, one traditional, part (a). Same instructor, different pedagogies.

Figure 8

The results of the general rubric applied to two algebra-based physics sections, one INQ, one more traditional, part (b). Same instructor, different pedagogies.

Figure 9

The results of the general rubric applied to four algebra-based physics sections, one small (24 students) INQ, one traditional interactive engagement (80 students) in 2010, two larger INQ (60 students) in 2017 and 2019, part (a). Same instructor for all sections.

Figure 10

The results of the general rubric applied to four algebra-based physics sections, one small (24 students) INQ, one traditional interactive engagement (80 students) in 2010, two larger INQ (60 students) in 2017 and 2019, part (b). Same instructor for all sections.