Patterns of multiple representation use by experts and novices during physics problem solving

It is generally believed that students should use multiple representations in solving certain physics problems, and earlier work in PER has begun to outline how experts and novices differ in their use of multiple representations. In this study, we build on this foundation by interviewing expert and novice physicists as they solve two types of multiple representation problems: those in which multiple representations are provided for them and those in which the students must construct their own representations. We analyze in detail the types of representations subjects use and the order and manner in which they are used. Expert and novice representation use is surprisingly similar in some ways, especially in that both experts and novices make significant use of multiple representations. Some significant differences also emerge. Experts are more flexible in terms of starting point and move between the available representations more quickly, and novices tend to move between more representations in total. In addition, we find that an examination of how often and when multiple representations are used is inadequate to fully characterize a problem-solving episode; one must also consider the purpose behind the use of the available representations. This analysis of how experts and novices use representations sharpens the differences between the two groups, demonstrates analysis techniques that may be useful in future work, and suggests possible paths for instruction.


I. INTRODUCTION
Physics education research ͑PER͒ has a long history of research into problem solving, especially as it pertains to novices ͑see Maloney 1 or Hsu 2 for reviews͒.Much of this research has investigated novice ͑and sometimes expert͒ use of representations during problem-solving episodes."Representations" is broadly used and can refer to whether the problem solver is representing surface features of a problem or the deeper physics. 3,4The term can also refer to the use of multiple representations together, such as pictures, free-body diagrams, and equations.Note that in this paper, we will use the term "representations" to refer to external representations, as opposed to mental representations.
Outside of physics, there has been considerable investigation of expert and novice use of representations during problem solving.In chemistry education, Kozma has studied the use of representations by experts in the workplace as compared to novices engaged in academic tasks. 5,6In geography education, there has been a study of expert and novice use of maps as representations. 7athematics education research has been especially prolific; see Ref. 8 for a collection.Schoenfeld's work is particularly relevant, as he broadened the study of expert-novice differences during problem solving to consider the metalevel activities that students engage in, noting such things as whether a student is engaged in goal-driven analysis or unfocused exploration. 9,10He found that experts are more likely to engage in careful planning, analysis, and reflection.We draw on Schoenfeld's analysis techniques in this paper and confirm the differences in metalevel problem-solving behavior between experts and novices in physics.
2][13][14] Less prior work on multiple representation use in problem solving examines how students apply representations to solve particular problems.Some early examples of this work found that expert and novice representations differ by whether they focus on surface or deep features of the problem ͑focusing on balls or pulleys rather than on force or energy principles͒. 4,15,16arkin and Simon have made significant contributions both to the study of expert and novice problem solving in physics 17 and to the study of representation use by experts and novices. 4,18They also note that one of the major differences between expert and novice problem solvers is the span and organization of the expert's domain knowledge, as this allows them to access many stored problem schemas.While we do not analyze our data in those terms, we consider our observations consistent with the idea that experts have a large and well-indexed knowledge base.Larkin and Simon also described a typical expert pattern of representation use, in which the solver constructs a pictorial or diagrammatic representation, followed by a physical representation ͑one that focuses on physical entities, such as a free-body dia-gram͒, and ending with a mathematical representation.Experts are especially skilled in making transitions between these representations.In the present study, we did not observe the novices to focus more on surface features than experts, nor did we see experts to be substantially more likely to use physical representations.We do, however, make a related observation that novices and experts differ significantly in how they use the representations that they produce, even when they produce similar collections of representations.
More recently, in some of our previous collaborative work, we have confirmed that instruction rich in multiple representations fosters student use of multiple representations during problem solving ͑pictures and free-body diagrams in this case͒, and that this fostering of representation use can happen in substantially different environments. 14We also found that while use of multiple representations during problem solving was associated with success, it was not a guarantee of success.These findings and other work do much to clarify the coarse differences between expert and novice representation use when solving physics problems, but some significant questions remain at a finer grain.Will experts and novices differ in the order of representations used or in how often and how quickly they move between them?To what extent does the result depend on whether the subjects are working with representations that they have generated themselves vs representations that were provided?Finally, at the metalevel, what differences exist regarding why and how ͑including to what ends͒ experts and novices use representations?Having a clearer picture of how experts and novices differ in their approaches to multiple representation problem solving will allow us to better bridge the expert-novice gap with instruction.
In our study, we begin to address these issues in detail.We interview 11 undergraduates enrolled in algebra-based introductory physics and 5 physics graduate students, providing novice and expert samples ͑with respect to the topics studied; the "experts" may well be novices in other areas͒.In these interviews, the subjects solve a variety of multiple representation based tasks in kinematics and electrostatics.The tasks vary in difficulty, such that some are problems in the sense that solution paths are not immediately apparent, while others are exercises for these subjects. 19Nearly all subjects, expert and novice alike, encountered at least one task that was a problem and at least one that was ͑for them͒ an exercise.We then perform a fine-grained analysis of these interviews, with the goal of characterizing the major differences and similarities between the two samples in terms of success rate, frequency and sequence of representation use, and purpose toward which these representations are applied.To these ends, we present a variety of data analyses.The first of these are timing data that take into account the amount of time spent on various tasks.For example, we measure how often and when students use representations during problemsolving episodes.Second, we find it useful to attend to sequential data, or data that focuses explicitly on the order in which students use particular representations.Third, we find that we need to characterize the purposes for which these representations are used.For this, we adapt a classification scheme used by Schoenfeld 9,10 with such activity categories as analysis, implementation, and verification.
We should emphasize our perspective regarding representational competence, problem solving, and context.While it is possible to consider skill with a representation divorced from the applications in which it is used, we do not take that approach.Rather, we consider representational competence to mean competence with representations in applied contexts. 20As a result, we make no attempt here to separate ͑for example͒ our subjects' abilities with graphs in general ͑whatever that may mean͒ from their success in using graphs to solve physics problems.
Our data presentation begins with three case studies of individual students solving a problem, who we classify as a weak novice, a strong novice, and an expert.These students introduce the kinds of data available and serve as a summary of the 11 novice and 5 expert students that we have analyzed.These in-depth case studies also demonstrate the utility of the analysis techniques we apply to make sense of representation use during problem-solving episodes.These analysis tools are extensions of or combinations of previously established techniques.
Based on the analysis of all the subjects, we present two categories of results.First, we describe quantitative similarities and differences between the experts and novices with respect to representation use.As one would expect, the experts solve their problems more quickly, often using the same set of representations as novices but in a shorter time.Surprisingly, however, the novices interviewed do not show the reluctance to use a variety of representations that one might expect.Indeed, their solutions were often more complex in terms of number of translations between representations.This may be partly a consequence of the nontraditional nature of their introductory physics course.We have observed in other work that students in representationally rich PERinformed courses make significant use of multiple representations when solving problems. 14,21Second, we consider the behaviors our subjects engage in while using multiple representations.Here, experts differ noticeably from novices, with experts exhibiting more careful analysis and self-checking, less weakly directed, unplanned work and more flexibility in terms of starting point.
Taken together, our results allow us to contribute to the picture of what distinguishes a novice from an expert in terms of multiple representation use.These findings clarify what features do not necessarily distinguish novices and experts, such as the simple presence of multiple representation use.

II. METHODS: POPULATION AND PROBLEMS USED
Our problem-solving interviews were clinical, in the style of diSessa et al. 22 The interviews lasted between 40 and 60 min and were videotaped.All participants volunteered after solicitation through mass electronic mails to the introductory physics course and to the graduate student list.Participants were paid for their time and were selected on a firstcome-first-serve basis.The expert and novice participant pools both had physics grades and backgrounds that were representative of the groups from which they were drawn.
Six of our novices were drawn from the first-semester large lecture introductory algebra-based physics course in the fall of 2005, which we refer to as Physics 201.The other 5 novices ͑for a total of 11 novice interviewees͒ were drawn from the second semester of this course in the spring of 2006, Physics 202.All of the interviews took place late in the semester, and so the novices had at least completed instruction in the topics covered by our study tasks.Our five expert problem solvers were physics graduate students, usually in the first year or two of their program.Entering graduate students at the University of Colorado generally exhibit near mastery of introductory physics.All graduate teacher assistants are given the Force and Motion Conceptual Evaluation ͑FMCE͒ 23 and Brief Electricity and Magnetism Assessment ͑BEMA͒ 24 on arrival and typically score 85%-90% on the BEMA and nearly 100% on the FMCE.The 201 and 202 population were made up mostly of life science majors, with many students on a premedical track.The courses included lecture and recitation sections, with approximately 250 students per lecture section and 25 per recitation section.The course was taught by an experienced member of the physics education research group and made heavy use of peer instruction 25 and personal response systems ͑clickers͒.More details about the course can be found in Ref. 14.
The three categories of students studied ͑novices from 201, from 202, and graduate students͒ solved three sets of problems.The 201 novices solved what we will refer to as the car problem.The students were given sets of representations of the motion of a car.This collection included a set of graphs of position versus time, a set of graphs of velocity versus time, a set of Flash animations depicting a moving car, and a set of written descriptions of a moving car.The students were instructed to make as many groups as possible of members from the various sets; that is, they were told to select position graphs, velocity graphs, animations, and written descriptions that all corresponded to one another.They were also told that not all members of each set would be used in all groups and that it was possible to find partial groups of less than four elements.In general, we took care to make it difficult to construct groups based on elimination or other such strategies.There were six Flash movies, providing a rough span of the motions present in this problem.In movie A, a car enters from the left of the screen, slows down, reverses direction, and exits, showing constant acceleration.
In movie B, the car starts on the left side at rest and constantly accelerates until it goes offscreen to the right.In movie C, the car drives from left to right at a constant speed.In movie D, the car comes in from the left, suddenly stops, remains motionless for a moment, and then exits to the right with constant acceleration.In movie E, the car is motionless.In movie F, the car moves from left to right, slowing down without coming to a halt.Corresponding motions ͑and oth-ers͒ are represented in the graphs of position and velocity and in the written descriptions.Figure 1 displays a sample set of representations, including a snapshot of the Flash movie, a position graph, a velocity graph, and a written description.The members of this sample set all correspond to the same motion and answer group.
The second set of problems are the five electrostatics problems seen in Ref. 14 and were solved by the Physics 202 novices.All of these problems involved numerically calculating either a force or a charge.One explicitly required the production of a free-body diagram ͑FBD͒ as part of the answer, while all five ͑especially the fifth or challenge prob-lem͒ were made easier by drawing a picture and/or an FBD.In Fig. 2, we see the first four electrostatics problems.Figure 3 has the challenge problem and an example solution.
The expert problem solvers solved all of the problems given to the 201 and 202 novices, as well as one problem designed to be challenging for an expert.The expert problem is the pulley problem used by Larkin in Ref. 4. The problem statement is presented in Fig. 4.This problem is extremely difficult without constructing a picture and a free-body diagram.The correct ratio is W 1 : W 2 =1:5.
FIG. 1. Four representations of the motion of a car ͑a snapshot from a Flash movie, a graph of position versus time, a graph of velocity versus time, and a written description͒.The students interviewed were required to make sets of these representations that all corresponded to the same motion.
In summary, there were three groups of interviewed students.Six novices from Physics 201 solved the car problems that required them to group representations that were provided to them.Five novices from Physics 202 solved the five electrostatics problems that had no representations provided beyond a written problem statement.Five expert physics graduate students solved all of the novice problems and also solved a more difficult problem involving a system of pulleys.

III. METHODS: CODING
All of the interviews were coded in two main ways.First, we coded representation use as a function of time.We divided the interview episode into 10 s blocks and noted which of the available representations ͑pictures, FBDs, written language, math, movies, position graphs, and velocity graphs͒ students made use of or made reference to in each time block.It was possible for more than one representation to be present in each block or for no representations to be coded in a particular block.This procedure was quite similar to one performed and vetted in previous studies, 14,26 and we performed no additional reliability tests.
The parts of the interviews corresponding to the solution of the electrostatics and pulley problems were coded in a second way, as well.Here, we coded for the kinds of activities students engaged in, such as planning and implementation.We adapted the activity categories and rubric from Schoenfeld, 9,10 with input from the paper authors and a member of our research group unrelated to the project.The activity rubric follows.
Reading: Reading the problem statement, either aloud or quietly.This includes silence following the initial reading when accompanied by a gaze or gesture in the direction of the problem statement.Rereadings of the statement on the order of 3 -5 s for the purposes of other activities ͑such as analysis͒ are not coded as reading, but long rereadings are.
Translation: Taking information directly from the problem statement and rerepresenting it.This includes writing numerical data or the quick construction of a diagram on which to place data from the problem statement.This does not include substantial work independent of the problem statement.
Analysis: Represents a directed, systematic attempt to understand or solve the problem.It can include constructing supplementary representations such as free-body diagrams or pictures once the reading and translation phases are over.It can include talking out loud about their understanding of the problem or calculating intermediate quantities that need to be obtained before a final answer can be reached.Reasoning does not have to be correct to be coded as analysis.
Exploration: A less-structured version of analysis.The student is searching for options or trying things out with little direction.Examples include a student searching through equations in the book, listing remembered equations without making use of or reference to them or cycling through their previous work out of apparent inability to proceed further.
Planning: Explicitly planning future moves ͑this usually means saying it out loud͒.A student will say "Ok, I need to draw a free-body diagram, then I'll balance forces, then I'll solve for my variable."When a student begins implementing their plan or returns to analysis, this phase ends.
Implementation: Differs from analysis and exploration in that it is strictly process oriented or "crank turning," generally mathematical, but including any algorithmic crank turning with little clear awareness of what students are doing or why ͑for instance, drawing a picture because "that's what you do," even if a student never refers to it again͒.Note that this does not require students to be correct; if a student is on an inappropriate path, this can still count as implementation.
Verification: Follows the discovery of an answer ͑right or wrong͒ and involves taking some kind of step to check the correctness of the answer ͑a final or intermediate answer͒.If a student decides outright that an answer is wrong and begins work again, this is analysis, exploration, and/or implementation rather than verification.This includes reflection on A small (100 g) metal ball with +2.0 μC of charge is sitting on a flat frictionless surface.A second identical ball with -1.0 μC of charge is 3.0 cm to the left of the first ball.What are the magnitudes and directions of the forces that we would have to apply to each ball to keep them 3.0 cm apart?
A sphere of 0.3 kg is charged to +30 μC.It is tied to a second chargeable sphere by a 20 cm rope, and the spheres sit on a frictionless table.If the rope will break at 4.8 N, what charge needs to be on the second sphere to cause the rope to break?Hint: It may be useful to draw a force diagram.
A frictionless metal cart is being held halfway between two stationary charged spheres.The cart's mass is 2.5 kg and its charge is +5.0 μC.The left sphere has a charge of +1 μC and the right sphere has a -2 μC charge.The two spheres are 20 cm apart.At the instant the cart is released, what is the magnitude and direction of the total force on the cart?Refer to the included diagrams for help.whether the answer obtained is reasonable or expected.
As before, we divided the interview into 10 s segments and coded each block according to which activity the students were involved in.No block was coded as having more than one kind of activity unless that block contained a transition from one kind of activity to another.To test for interrater reliability, an author and one unrelated researcher each coded the same problem-solving session.The coders agreed 86% of the time.An example Schoenfeld-style plot is shown in Fig. 5.The episode displayed is a novice attempt to solve the electrostatics challenge problem.The student's attempt involved a lot of equation hunting and generally unfocused behavior and was ultimately unsuccessful.During our analysis, we performed this coding for each of the subjects that solved the five electrostatics challenge problems ͑expert and novice͒, as well as for the episodes in which the experts solved the pulley problems.

IV. METHODS: TIMING DATA
With representation use coded, we then calculated a number of numerical parameters.These included time spent per problem, number of representations used per problem, the density of different representations used per unit of time, and the number and density of transitions between representations.Transitions were identified and counted in a manual examination of the coded data, which included a rereview of the videos.
We also produced graphs of the representations being used as a function of time.Figure 6 shows an example graph, showing a 202 novice's solution of the electrostatic challenge problem.The bars show usage of each of the indicated representations, with time in minutes displayed on the bottom axis.

V. METHODS: SEQUENTIAL DATA
Our timing data are useful for seeing representation use as a function of time but are less useful at capturing the character of a problem-solving episode at a glance.An alternative is to depict the representations being used in the order that they are used.Figure 7 shows two such examples.In the first of these, we see a novice student's correct solution to the first of the five electrostatics problems.The student reads the written description of the problem, works with a set of equations, and draws a picture to clarify the direction of the force calculated.Once he has moved on from a particular representation, he does not return to it ͑this is not necessarily characteristic of success in general͒.In the lower diagram, we see another student's unsuccessful attempt to solve the same problem.The double-headed arrow represents a very close coordination between two representations, in this case, the problem statement and a picture that he draws while continuously referring to the problem statement.The student then draws an FBD, writes some equations, refers back to the FBD, draws a picture, does more math, returns to the problem statement, and takes a look at the picture before giving up.
This visualization, which we will refer to as a sequence diagram, provides a quick and easy sense of the complexity or linearity of a student's solution.For the purposes of comparing many students, it is useful to define a numerical parameter that expresses the complexity of the diagram.An obvious first choice for this complexity parameter would be the number of transitions indicated by the diagram.However, this choice has two difficulties.First, it treats close coordination of representations the same way as a switch between representations, and second, it over-represents quick, repeated transitions between the same two representations ͑a student can easily look back and forth between two representations half a dozen or more times in the span of a few We have three pulleys, two weights, and some ropes, arranged as follows: 1) The first weight (W 1 ) is suspended from the left end of a rope over Pulley A. The right end of this rope is attached to, and partially supports, the second weight.
2) Pulley A is suspended from the left end of a rope that runs over Pulley B, and under Pulley C. Pulley B is suspended from the ceiling.The right end of the rope that runs under Pulley C is attached to the ceiling.
3) Pulley C is attached to the second weight (W 2 ), supporting it jointly with the right end of the first rope.seconds͒.To solve this sampling problem, we count the number of transitions between representations, counting only one transition per 10 s block for periods of quick back-and-forth transitions and counting one transition for a move into a period of close coordination and one transition for a move out of a period of close coordination.This count provides the number we call the complexity parameter, which is displayed for each of the problem solutions shown in Fig. 7.Note that this complexity parameter serves as a modified count of the number of representation transitions present.

VI. DATA AND ANALYSIS: CASE STUDIES
Before turning to aggregate data, we provide examples of thorough characterizations of problem-solving episodes in order to demonstrate the utility of these analyses.We will consider the solutions of the electrostatics challenge problem for three sample students, using pseudonyms in all cases.The first of these, Carrie, we considered to be one of the weaker novices interviewed, and she was unsuccessful in solving the problem.The second of these, Sam, was the strongest of the novices interviewed and was successful in her attempt.Finally, we consider Jim, who was in our opinion the strongest expert.He solves the problem correctly and very quickly.In each case, we will begin by describing the student's performance overall and our impression of the student.We will then examine the student's solution to the challenge problem in detail.

A. Case study: Carrie
Carrie struggled with the electrostatics problem set, getting none correct.In general, Carrie was not completely ignorant of the appropriate solution techniques, remembering such things as Coulomb's law and F = qE, but her recollections were often piecemeal, and her attempts to fill in gaps in her understanding were not successful.Among other things, she remembered Coulomb's law as having a 1 / r dependence, which contributed to her errors on the first three problems.In some ways, we would describe her as the stereotypical novice, applying learned procedures and equations with little understanding.
In Fig. 8, we see the results of three different analyses of Carrie's solution to the electrostatics challenge problem.One Carrie begins by reading the problem and translating the description into two sketches and a free-body diagram.She then arranges the charge and distance information into Coulomb's law without an intermediate symbolic step and remembers out loud that the functional form of Coulomb's law is like that of the law of gravity.Next, she draws revised versions of her picture and free-body diagram, adding more labels and information.Importantly, her picture of the setup is incorrect, with positive and negative charges hanging on strings as if they were repelling.
At 4:30 into the problem, Carrie looks briefly at the problem statement, and then re-examines her picture and her freebody diagram.She says that she is confused as to why the charges are not touching if one is positive and the other is negative.Unable to resolve this, she dismisses the issue and returns to Coulomb's law, beginning a long episode in which she tries to decide which elements of her picture fit into Coulomb's law.She eventually gives up.
We should revisit our remark about Carrie being a stereotypical novice, as this label has many possible interpreta- tions.Carrie is not hesitant to use nonmathematical representations and is at times careful in coordinating them.However, this is true of all students studied and may be a result of the representation-rich course she was a member of ͑the University of Colorado course featured in Ref. 14͒.Our anecdotal impression was that Carrie was drawing pictures and free-body diagrams because that is the norm and that only once did she turn these representations toward sense making and analysis ͑4 : 30-6 min on the charts͒.Indeed, most of her time spent was coded as "implementation," during which she appeared to be manipulating pictures, diagrams, and equations mechanically.Ultimately, the problem was too complex to be solved given her inability or unwillingness to engage in deeper and/or sustained analysis.

B. Case study: Sam
Sam was the strongest overall of the novices interviewed with the electrostatics problems.Sam answered problems 1, 2, and the challenge problem correctly and missed problems 3 and 4 due to misplaced factors of 10.Sam appeared comfortable working symbolically, usually made sound inferences, and even used unit analysis to aid in solving problem 2.
In Fig. 9, we diagram Sam's solution to the challenge problem.She began by drawing a picture based on the problem statement, coordinating extremely closely between the two ͑looking quickly between them, marking points of correspondence with her fingers, and so forth͒.She briefly revises her picture and then begins drawing a free-body diagram and thinking out loud about which forces should go where ͑with this, in part, prompting us to code a period of analysis͒.
Three min into the problem, Sam looks back at the problem statement and notes that the picture she has drawn shows the charges repelling, when they are of opposite sign.She explicitly wonders how this could be correct, and whether she's made a mistake.After more analysis and reference to her picture and to her force diagram, Sam draws the correct picture and draws an updated free-body diagram on top of it.She makes a brief statement about her intentions, coded as planning.
Near the 5 minute mark, Sam says that she does not know which of the available forces she should use to get the tension or how to combine them if necessary.She begins a period of thinking out loud without clear reference to representations on the paper ͑but with considerable gesture, a kind of representation not considered in this study 27 ͒.The period of exploration indicates an ultimately fruitless examination of her class notes.At 8 min, Sam spends more time gesturing and updating her free-body diagrams while thinking about which forces apply.She eventually realizes the solution, which she spends the last 2.5 min implementing, carefully referring back to the problem statement to verify the numbers she is using in her symbolic equation.

C. Case study: Jim
Jim was a first-year graduate student, but was noticeably stronger than any other expert interviewed.He solved all the problems quickly and correctly and then solved an additional reserve problem.Despite the apparent easiness of many of the novice tasks ͑from his perspective͒, he still drew a picture on all but problem 2 of the electrostatics set.
In Fig. 10, we see representations of Jim's solution to the electrostatics challenge problem.After thoroughly reading the problem ͑nearly a minute of reading before taking any other actions͒, he drew a correct picture of the charges on strings.He was nearly unique in this regard, as some of the experts still had some difficulty in translating the problem statement into a picture without getting it backward temporarily.He then set up a block of equations before rereading the problem.Just after the 2 min mark, he said that he was trying to figure out how far apart the strings should be.After some deliberation, he concluded that it did not matter.He then set up a free-body diagram, checked it against his first block of equations, wrote a second block, and solved for the necessary force.
Our primary impression from Jim was one of efficiency with algorithms and mastery of the relevant concepts.Still, the problem was complex enough that his use of pictures and free-body diagrams did not appear to be token or out of habit, rather, he appeared to be using both of these in making sense of the problem and possibly in nonverbalized selfchecks.Note that there was no period coded as verification, as we never perceived him to be engaged only in checking his answer.This does not rule out the possibility of selfchecking over the course of the solution.

D. Case studies: Comparisons
These three students represent the full spread of subjects interviewed, including a weak novice, a successful novice, and the most expert problem solver interviewed.At least one major similarity presented itself.All three of these subjects, much like all the others, made considerable use of multiple representations in their problem solutions.This finding differs from early evidence of introductory students using multiple representations only rarely, 28 but is consistent with more recent observations that students use pictures and free-body diagrams quite often in representation-rich PER-informed courses. 14,21As a result, while the complexity parameter we use helps distinguish the expert and novice groups in aggre-gate, we find that without checking on the total time elapsed, it is difficult to identify the weak novice, strong novice, and expert using only the displays of what representations were being used and in what order ͑the sequence diagrams and representations vs time chart͒.These students become most distinguishable when we consider how they used representations.As seen in the Schoenfeld-style plots, the expert and successful novice spent time using the representations to make sense of the physics, while the unsuccessful novice appeared to be drawing pictures and free-body diagrams out of a sense of requirement and not toward any particular purpose.
This sample of case studies is representative of the data as a whole.Examination of solution sequence diagrams and representation vs time charts is not nearly as useful in separating novices from experts as is combining these data with Schoenfeld-style plots representing the kinds of activities students engaged in.

VII. DATA: AGGREGATE
The tools described in Secs.II-V provided a large data set.As a result, it will be necessary for us to be selective in our presentation.The data we show here are chosen to be a subset that is both concise and descriptive.

A. Data: Correctness and timing
In Table I, we see student performance on all of the problems solved.The "Cars" column shows the number of correct and incorrect representation groupings made by the expert and novice students, in a correct/incorrect format.Most of the difference between experts and novices here can be traced to the fact that only one of the novices attempted to make a group that did not include a movie, whereas all of the experts made at least one subgroup that did not include movie, using a position or velocity graph as a starting point.
The electrostatics columns for problems 1-4 and the challenge problem show the number of students answering correctly/incorrectly on the first four electrostatics problems and the electrostatics challenge problem.Recall that the novices solving the electrostatics problems are not the same novices that solved the car problems, but that the same experts solved all problems.The last category shows the number of experts answering correctly/incorrectly on the expert-only pulley problem.Overall, the experts were very successful.One expert missed electrostatics problems 1 and 2 by incorrectly recalling Coulomb's law, writing it with a 1 / r dependency instead of 1 / r 2 .All other expert solutions were correct until they reached the pulley problem, at which point three were successful and two were unsuccessful.
The novices were reasonably successful on the car problem, forming an average of one incorrect grouping per person.This was frequently a grouping involving movie F, in which a car slowed down constantly without coming to rest.Despite the speed of the car upon exiting the screen being approximately half of the initial speed many of the novices perceived it to be constant speed, or had trouble deciding.In contrast, only one of the experts spent significant time considering this point, and that expert resolved the difficulty correctly.The fact that experts and novices differed so strongly in terms of their qualitative perception of the motion is surprising but is not unprecedented.This is perhaps related to the differences in readout strategies 29 observed by Mestre and co-workers between more and less expert students. 30,31n the electrostatics data, novice performance trends downward as we move across Table I from problem 1 to the challenge problem.No students solved problem 4 correctly, and one out of five solved the challenge problem correctly.This was expected and is consistent with the free-response data from a previous work. 14able II shows the average time taken by the experts and novices to solve the various problems.For the car problem, both experts and novices attempted groupings based on all of the movies, and so we display the time taken to make that set of groupings ͑5.8 min for experts and 14.6 min for novices͒.The other columns show the time taken for electrostatics problems 1-4 considered together, the challenge problem, and the expert-only pulley problem.Generally, experts took 40%-60% of the time required by novices to solve the same problems.Note that on the pulley problem, one expert had a very fast and correct solution ͑4 min͒, while all others took considerably longer.

B. Data: Representation use
Table III has a summary of the overall representation use by the students solving the problems studied.In the first two columns, we see the average complexity parameter associated with the sequence diagrams for the various problems.Low parameters represent a solution in which students used only the minimum set of representations in sequence, without revisiting representations during the solution.High pa-rameters represent a more complex, iterative approach in which students moved frequently among the representations available ͑whether these representations were provided or were student generated͒.Complexity is generally higher for the novice solutions overall.
For the electrostatics problem data, there are no significant differences in solution complexity except for problem 4 and the challenge problem.The novice solutions to the challenge problem are approximately as complex as the expert solutions to the pulley problems ͑parameters of 15.8 and 17.0, respectively͒.Furthermore, there is no significant difference between the average complexities of the correct and incorrect novice solutions ͑not shown͒.
In the second two columns, we present an alternative method of characterizing representation use.There, we see the time density of transitions between representations, in transitions per minute.This time density of transitions provides a measure of how quickly the problem solver was moving back and forth between the representations available.Experts show no clear and consistent variation in transition density from problem to problem when considering the electrostatics and pulley problems.The experts and novices both show much more frequent transitions on the car problems, representing the fact that the available representations were provided for them instead of needing to be produced on the fly.
The representation transition data are intriguing: as compared to novices, experts show a higher density of transitions in all categories, consistent with the expectation that experts will be using multiple representations for these problems and solving them quickly.It is worth noting that while experts would not necessarily need to create other representations ͑pictures and free-body diagrams͒ to make sense of these novice tasks, it has been our observation in this study that these expert physicists generally did make use of multiple representations in these cases.

C. Data: Solution strategy
On the problems studied, there was some flexibility as to which representations the students could start with on each problem.For instance, after reading the electrostatics problem statements, students could draw a picture, draw a freebody diagram, or write equations.On the car problems, we TABLE II.Average time ͑in minutes͒ for experts and novices to complete the interview tasks."Cars, A-F" column shows the time to complete all groupings involving movies A-F.Note that the experts produced 25% more car groupings than the novices on average."Prob.1-4" refers to the first four electrostatics problems.took care not to present any one of the four available representations as the intended starting point, so that students were forced to choose their own starting point.
On the electrostatics problems, experts and novices were quite consistent.Of the five novices that completed all five problems, we see only two instances where the novice started with an equation ͑one on problem 1 and one on problem 2, not the same student͒.In all other cases, the student started with a picture.Experts were similar: on nearly every problem, including the pulley problem, the expert began with a picture immediately after reading the problem statement.One expert began problem 1 with an equation and another started with an equation on problems 1-3.
On the car problems, novices were very self-consistent.Three of the novices began every grouping by picking out a movie to use as a centerpiece, while the other three began nearly every grouping with the written description.Among the students starting with written descriptions, the only exception was one student who started five groups with a written description and began the sixth with a movie.In short, novices perceived either movies or the written descriptions to be the natural starting points and did not deviate from their choices.
Experts were considerably more flexible.Experts started their problems using all four of the available representations ͑written, position graphs, velocity graphs, and movies͒.Only one of the experts chose a starting representation consistently ͑the written one͒, and none of the other four deviated less than twice from their primary choice.In total, we have 13 expert groupings that began with movies, 13 that began with written descriptions, 7 that began with velocity graphs, and 5 that began with position graphs.
We can compare expert and novice flexibility in at least one other way.When solving the car problems, novices always finished one group before moving on to another and very rarely returned to a finished group, doing so only when a later group directly conflicted with a previous group.One of the experts departed radically from this pattern, making many of his groups simultaneously, in parallel.A second expert completed as many three-representation groups as possible before going back and adding in the movies last.The other three experts generally finished groups in sequence but were relatively comfortable with returning to a completed group or looking ahead to a group yet to be made.

D. Data: Schoenfeld-style codings
Next, we consider the purposes toward which individuals applied the representations they used.We focus mainly on the electrostatics problems in which students had to generate their own problem representations.The graphs in Fig. 11 show the fraction of the codeable time intervals occupied by each of the activities listed for experts and for novices.The biggest differences between the two groups appear in the reading, analysis, and exploration categories.Experts spend a greater fraction of their time reading the problem, although we believe this to be an artifact of their generally shorter solution times.More significant is the time spent in exploration and analysis by the novices.Analysis represents a di-rected, systematic attempt to more fully understand the problem.Analysis generally is explicitly goal oriented, trying to figure out some specific aspect of the problem.Exploration represents less focused behavior, where the student is searching for options or trying things out with little direction.When behavior is coded as exploration, it is generally not clear what subjects expect to find through their efforts.On average, the experts spent 43% of their time on analysis and 1% of their time on exploration, while the novices spent 25% of their time on analysis and 16% of their time on exploration.In this sense, experts were significantly more likely to use the available representations in what we might consider more productive ways, while the novices were more likely to demonstrate unfocused representation use.This is consistent with our qualitative impression from the interviews: novice students, especially weaker ones, often produced pictures, diagrams, or equations without then being able to use them to advance their solution.
There is a possible confounding factor: the electrostatics problems were very challenging for the novice students but were relatively easy ͑though not trivial͒ for the expert students.Thus, the observed differences could conceivably have been an effect of the problem difficulty: to the experts, the electrostatics problems may be exercises rather than problems, as defined by Mayer. 19In response, we also coded expert behavior on the pulley problem, which was designed to be an actual problem in this sense for the experts.On the pulley problem, experts demonstrated similar success to the novices on the later electrostatics problems.Between the reduced success and our qualitative observation that the experts usually did not know how to proceed without considerable thought, we are comfortable in concluding that the pulley task represented a problem.The data for the expert pulley problems are in Fig. 12.On problems that should represent actual problems for these expert physicists, the experts produce the same analysis-heavy distribution of representation use that they do on the exercises coded for Fig. 11.

VIII. DISCUSSION
In this study, our goal was to more thoroughly characterize the differences and similarities between expert and novice use of representations while solving physics problems.Our interviews with Carrie, Sam, and Jim are representative of all the interviews conducted and generally reflect the aggregate data.Our analysis of the aggregate data suggests key differences between expert and novice problem solvers and at least one notable similarity.One would reasonably expect significant differences in success and speed, which we did see.However, one might also expect differences in willingness to use multiple representations, which we did not see.Furthermore, we did not observe some of the classically expected expert-novice differences.We did not observe the novices to focus more on representing surface features than experts 3 nor did we see experts to be substantially more likely to use physical representations. 4This may have been a feature of the kinds of problems we used, as the electrostatics problems are very typical problems where students are likely to be familiar with the expected solution routine.Alternatively, it may have been a feature of the kind of instruction the novice students have received.We have previously established that the introductory courses that these students took are representationally rich and that the students use multiple representations quite frequently even when it is not required. 14e had two classes of problems in this study: the car problems, in which a variety of representations were provided for the students, and the electrostatics problems that had only a written problem statement.The former required the coordination of available representations, while the latter required generation of additional representations.Generally, the two classes of problems did not lead to major differences in the expert-novice comparisons.There was one exception of note.The electrostatics problems were of a style familiar to the students, and they ͑expert and novice alike͒ tended to rely on a standard pattern of representation use ͑read the problem, draw a picture and/or free-body diagram, and solve equations͒. 28The car problems were designed to be less familiar and potentially much more flexible, and here experts and novices did differ in terms of representational flexibility.Novices always worked from the same starting representations in solving the car problems and, across all novices interviewed, only used two of the available representations as starting points.Novices also solved one group fully before progressing to another, almost never revisiting old answers or leaving a partial group early.Experts used all four available representations as starting points, used different representations as starting points within the same task, and were more likely to work in a piecemeal or iterative fashion when it suited them, exhibiting greater flexibility in their problem solving with no associated performance cost.
Perhaps the clearest indicator of novice versus expert status ͑besides correctness and time to finish͒ was the breakdown of how the students used their representations.Experts consistently engaged in considerably more analysis.Novices were more likely to behave mechanically or algorithmically and to produce multiple representations without being able to make much use of them.This is consistent with Schoenfeld's observations in mathematics. 9Interestingly, experts showed similar representation use on the novice problems that they solved and on the pulley problem that was chosen to represent a true problem for these experts ͑as opposed to an ex-ercise͒.
If these results are widespread and repeatable, they are significant.As we have seen before, 14,21 PER-informed courses can be successful at getting students to use a variety of canonical representations while solving physics problems.However, students may not be learning why they are using these representations or how to use them to maximum effect.This is reasonable since metalevel skill tends to be slow to develop. 19Furthermore, classes almost never teach metalevel problem-solving skills ͑including metarepresentational competence 32 ͒ explicitly ͑either as standalone skill sets or, more likely, in the context of solving actual problems͒, which means that if they are learned, they are picked up informally over many courses.We speculate that attending to metalevel skill sets ͑like knowing what different representations are useful for͒ may result in more expertlike use of problem-solving time and greater success; however, it is not clear how to teach these skills formally or whether they can be consistently taught to novices at all in the short time available in introductory physics courses.Follow-up work on these metarepresentational issues would likely be productive.

IX. CONCLUSIONS
In this paper, we have presented techniques ͑or adaptations of existing techniques͒ for analyzing multiple representation problem-solving episodes to document how expert and novice students use representations in problem solving.We have applied these to representative case studies drawn from PER-influenced populations, which extend the prior work in the domain of problem solving.We have also provided data describing the differences between expert and novice problem solvers when handling multiple representations.Some of our results were expected: experts were more successful in solving problems that required the use of multiple representations, finished faster, and moved more quickly among the representations available.However, we were surprised to find that these novice problem solvers were just as likely to use multiple representations extensively in their solutions and that they used very similar selections of representations.This may be because doing so has become the norm in these representationally rich physics classes, and so they draw pictures and free-body diagrams regardless of their understanding of why.Novice solutions also included on average more translations between the different representations at hand, confirming that they were, in fact, making significant use of these representations, regardless of correctness.The large number of translations may also be reflective of the substantial exploratory behavior seen in our codings.
Considering only the number, kind, and correctness of the representations used did not fully characterize the differences between the experts and novices.We coded and analyzed the behaviors that subjects engaged in while using these representations, which led to our second set of results.When comparing experts and novices in terms of how they used representations, the time spent in different activities was quite similar except in the categories of analysis and exploration.Experts spent a greater fraction of their time pursuing specific goals or subgoals even when they did not know exactly how to proceed, as with the pulley problem.Novices still some spent time in this manner ͑which is encouraging͒ but were more likely to engage in exploration with no clear purpose, perhaps hoping to strike upon a correct approach.
It may be productive in future work to focus more directly on the order of representations used by experts and novices.In the more traditional electrostatics problems, both groups followed a typical picture-free-body diagram-equation path, but in the less-familiar car problems, the experts and novices differed significantly with respect to which representations they started with and in what order they were handled.Further and finer-grained investigation of these differences in representational flexibility may yield significant results.
Our observations of expert and novice problem solvers have suggested tentative paths for instruction and future study.It may be possible to teach novices greater awareness of how and why an expert uses representations during problem solving.If so, this could lead to more productive use of student time when working on problems with increased performance and understanding as a result.
(not shown)A 100 gram ball has a charge of +40 μC.The ball is dropped from a height of 2 m into a 7000 N/C electric field pointing up.Draw a diagram showing all the forces involved in the problem, and calculate the magnitude and direction of the net force on the ball.

FIG. 2 .
FIG.2.The first four electrostatics problems solved in the interviews.

Find the ratio of W 1 to W 2 .FIG. 4 .
FIG. 4. The statement of the pulley problem given only to the expert group.
of these shows the representations she used as a function of time in minutes.The next shows a Schoenfeld-style diagram of the activities she engaged in on the same time scale.Last, we see a sequence diagram showing the sequence of particular representations used.

FIG. 7 .
FIG. 7. Sequence diagrams showing the order in which students used representations in two of their problem solutions.Above: A novice's correct solution to the first of the electrostatics problems.Below: A solution where a novice student did not reach the correct answer.The complexity parameter is defined in the text.

FIG. 11 .
FIG. 11. ͑Color online͒ Distribution of time spent in by experts and novices during their solutions to the electrostatics problems.The possibility of coding more than one behavior per interval leads to totals above 1.0.Error bars represent the standard error of the mean.

FIG. 12 .
FIG. 12. ͑Color online͒ Distribution of time spent in by experts during their solutions to the more challenging pulley problem.The distribution is similar to the expert distribution in the electrostatics problems of Fig. 11.

TABLE I .
Student performance on the study problems.The Cars column shows the number of correct/ incorrect groups and subgroups made.The electrostatics columns for problem 1-4 and the challenge problem show the number of students answering correctly/incorrectly for those problems.The pulley column shows the number of experts answering the pulley problem correctly/incorrectly.
FIG. 10. ͑Color online͒ Three codings of expert Jim's successful solution of the electrostatics challenge problem.

TABLE III .
Complexity parameter describing the representations used in student solutions, and density of transitions between representations ͑per minute͒ during those solutions.Organized by expert vs. novice and by problem.