Computer problem-solving coaches for introductory physics: Design and usability studies

The combination of modern computing power, the interactivity of web applications, and the flexibility of object-oriented programming may finally be sufficient to create computer coaches that can help students develop metacognitive problem-solving skills, an important competence in our rapidly changing technological society. However, no matter how effective such coaches might be, they will only be useful if they are attractive to students. We describe the design and testing of a set of web-based computer programs that act as personal coaches to students while they practice solving problems from introductory physics. The coaches are designed to supplement regular human instruction, giving students access to effective forms of practice outside class. We present results from large-scale usability tests of the computer coaches and discuss their implications for future versions of the coaches.

Figure 1

Screenshot from a type 1 coach (computer coaches student). The display shows a partially completed picture $①$. The computer specifies a step in the framework $②$ and asks the student to decide on the direction of a force $③$. The student’s decision $④$ is incorrect, and the computer provides feedback $⑤$. A red number to the right of each step $⑥$ indicates the number of incorrect responses the student made for that decision, while a checkmark indicates that the step was performed correctly the first time.

Figure 2

Screenshot from a type 2 coach (student coaches computer). The display shows a completed picture $①$. The student, acting as a coach, has decided on a step for the computer, in its role as a student, to perform $②$, but it is not an appropriate step at this point. The computer, in its oversight role, gives the student feedback $③$.

Figure 3

Screenshot from a type 3 coach (student works independently, computer gives feedback). If the student gets stuck solving a problem or enters an incorrect answer, the computer asks the student to decide where in the problem-solving process the difficulty might occur and to get help.

Figure 4

Example of a context-rich problem.

Figure 5

The Minnesota problem-solving framework [28].

Figure 6

Histogram of completion times for one particular type 1 coach. The average completion time was calculated using only the times that were within the main distribution (less than 70 min.).

Figure 7

Use of the computer coaches as a function of time throughout the spring 2013 course by students in each of the three usage groups. A database error prevents the calculation of student usage of the coaches in week 3, as well as causes the asymmetric error bars for the week 4 data. The solid vertical lines show the timing of the four in-class tests, which occurred at the end of weeks 4, 7, 11, and 15 of the class. The solid and dashed lines show the boundaries between different topics (labeled at the top of the graph). Usage by students in the $L$ and $H$ groups is relatively constant while usage by $M$ students changes dramatically between the energy and momentum sections of the course, in the middle of the week 10 homework. Spring break occurs between weeks 8 and 9 of the course.