Force, velocity, and work: The effects of different contexts on students’ understanding of vector concepts using isomorphic problems

In this article we compare students’ understanding of vector concepts in problems with no physical context, and with three mechanics contexts: force, velocity, and work. Based on our “Test of Understanding of Vectors,” a multiple-choice test presented elsewhere, we designed two isomorphic shorter versions of 12 items each: a test with no physical context, and a test with mechanics contexts. For this study, we administered the items twice to students who were finishing an introductory mechanics course at a large private university in Mexico. The first time, we administered the two 12-item tests to 608 students. In the second, we only tested the items for which we had found differences in students’ performances that were difficult to explain, and in this case, we asked them to show their reasoning in written form. In the first administration, we detected no significant difference between the medians obtained in the tests; however, we did identify significant differences in some of the items. For each item we analyze the type of difference found between the tests in the selection of the correct answer, the most common error on each of the tests, and the differences in the selection of incorrect answers. We also investigate the causes of the different context effects. Based on these analyses, we establish specific recommendations for the instruction of vector concepts in an introductory mechanics course. In the Supplemental Material we include both tests for other researchers studying vector learning, and for physics teachers who teach this material.


I. INTRODUCTION
Most physical concepts covered in introductory physics courses at the university level are represented by vectors.Therefore, a complete understanding of these physical concepts requires that students have a good grasp of basic vector concepts.In introductory physics courses, students need to learn how to solve vector problems with no physical context as well as problems with a physical context.A classic example is the addition of vectors.Students need to learn how to add vectors in the absence of context and also how to add forces that act on a body to obtain the net force.
In previous papers [1,2], we presented the design of a 20-item multiple-choice test [Test of Understanding of Vectors (TUV)] that evaluates students' understanding of vector concepts in the absence of physical context and that was designed in accordance with the recommendations by Beichner et al. [23,24].In those studies [1,2], we proved that the TUV is a reliable test with satisfactory discriminatory power.Using this test, we designed two shorter versions of 12 items each that are used in the present investigation: (1) a test with no physical context (referred to here as "TUV-12 items no context"), and (2) a test with three mechanics contexts: force, velocity, and work (known here as "TUV-12 items mechanics").
The main research question of this study is "What is the effect of mechanics context on second-year engineering students' understanding of vector concepts?"To answer the research question we defined two specific objectives in this study: (1) to understand the effect of context on students' understanding by comparing students' overall performance in two tests (TUV-12 items no context and TUV-12 items mechanics) as they were completing an introductory university mechanics course, and (2) to understand the effect of context on students' understanding by comparing students' performance in each of the items on both tests.Note that by vector concepts in this study we mean vector properties and vector operations including the skills to complete those operations.

II. PREVIOUS RESEARCH
The studies that compare students' performance in problems with no physical context with problems with a physical context [6][7][8]13,16,17,21] can be clustered in two subgroups: (1) studies that compare students' performances in one specific vector concept [8,13,16,17,21], and (2) studies that compare students' performances in many vector concepts [6,7].In Table I we present this classification of previous studies.
As shown in Table I, most of the studies that compare students' performance in problems with and without physical context were limited, because they only analyzed performance in one vector concept.Only Van Deventer's research [6,7] has compared students' performances on various vector concepts by using multiple-choice tests.However, the majority of the distractors (incorrect answers) on the tests were designed based only on interview results, so the analysis comes from a very small sample (eleven students).In the present investigation we based our questions on the TUV, which was designed following the steps recommended by physics education researchers and has proven to be a reliable test with satisfactory discriminatory power [23,24].Note that in the results and discussion sections we will relate the previous studies to the present study.

A. Test development
As mentioned in the introduction, the two tests used in the present study (TUV-12 items no context and TUV-12 items mechanics) were designed from the 20 items of the TUV [1,2].Table II shows the seven vector concepts evaluated in the tests, a description of the 12 items included, and the mechanics context used in each item.In the Supplemental Material [25] we present both tests.
As mentioned before, the TUV is a test with 20 items, while the two tests used in this study have only 12 items; therefore, eight items were not included.We did not include two items that evaluate students' understanding of the magnitude of a two-vector sum and one item that evaluates students' ability to calculate the direction of a vector written in unit vector notation, because item 1 and item 5 in the test used here evaluate the addition of vectors and the direction concept at a more fundamental level.We did not include two unit vector notation problems due to time constraints.
In addition, we also omitted the three items that evaluate the cross product operation because the torque concept, which is the most commonly used mechanics context for this operation, is not always presented and used in the TABLE I. Classification of previous studies that compare students' performance in problems with and without physical context.

Type of study Previous studies
Studies that compare performances in one vector concept Subtraction of vectors in 1D Shaffer and McDermott [21], Wang and Sayre [8] Addition of vectors in 2D Barniol and Zavala [13] Interpretation of dot product Zavala and Barniol [16,17] Negative scalar multiplication of a vector Zavala and Barniol [17] Studies that compare performances in many vector concepts Van Deventer et al. [6,7]  Calculation of dot product of vectors written in unit-vector notation Work introductory mechanics courses as a vector.Note, for example, the introduction of torque in the textbook of Serway and Jewett [26].In an early chapter of the book (Chapter 10: Rotation of a rigid object about a fixed axis) torque is presented without establishing its vector nature and students solve many problems with this definition.And only in the next chapter is the vector nature of torque established (Chapter 11: Angular momentum).An important fact to mention regarding the development of the two tests is that some items with noncontext were slightly modified from their original version on the TUV to conserve the strict isomorphism between the items of both tests.
As shown in Table II, we used three different contexts in the mechanics test: force, velocity, and work.The force concept is the central concept of the introductory mechanics courses; therefore, we decided to use this context in all of the items in which it had a physical meaning.As displayed in Table II, the six items that fulfill this criterion evaluated three vector properties (direction, magnitude, and components) and one vector operation (addition).Another three items that were included in the short version of the test were subtraction of vectors in 1D and 2D, and negative scalar multiplication of a vector (Table II).These three items all used the negative of a vector concept.
For these items, the force concept did not fully apply, so we decided to use the context of velocity, following the previous study of Shaffer and McDermott [21].Finally, in the three items that evaluated the dot product, we decided to use the work context, since it is related to this operation and it is the first concept with this characteristic that is used in a traditional introductory mechanics course.

B. Participants
The research was conducted in a large private university in Mexico.The participants in this study were engineering students who were finishing a calculus-based mechanics course.This course is the first of three introductory physics courses taken by students at this institution.The textbook used in this course was "Physics for Scientists and Engineers" by Serway and Jewett [26].Students also used the "Tutorials in Introductory Physics" by McDermott, Shaffer, and the Physics Education Research Group [27].
For this study, we administered the designed items twice to different students of different semesters of the same course.The first time we administered the two multiplechoice tests (see the Supplemental Material [25]).Figure 1 shows item 2 for both tests.In the first administration we focused on detecting the differences between students' performances in both tests.All scores and proportions presented in this article are from this administration.
In the second, we only tested the items for which we had found differences in students' performances that were difficult to explain, and in this case, we asked them to show their reasoning in written form.The five items were interpretation of dot product, calculation of dot product of vectors written in unit vector notation, negative scalar multiplication of a vector, and subtractions of vectors in 1D and 2D.In this second administration we focused on detecting students' reasoning.We present these students' reasoning when we try to explain these difficult-to-explain differences in students' performances taking into account the skills required for each particular vector question.
In the first test administration there were 608 students: 304 students solved the test with noncontext (TUV-12 items no context) and 304 students solved the test with mechanics context (TUV-12 items mechanics).In the second administration there were 203 students: 103 students solved some items from the TUV no context test and 100 students solved the associated items from the TUV mechanics test.In this second administration both populations were asked to provide their reasoning.Note that in both administrations the selection of students to present one or the other version of the test was made randomly from all different instructors.
The tests were administered in Spanish.

IV. STUDENTS' OVERALL PERFORMANCE ON BOTH TESTS
In this section we address the first objective of this study, that is, to understand the effect of context on students' understanding by comparing their overall performance on both tests.The medians of both tests are 7 (out of 12) which means that students who are on the median in both tests have difficulty answering 5 items correctly.Since both distributions of scores were not normal [Dð304Þ ¼ 0.104, p < 0.01; Dð304Þ ¼ 0.87, p < 0.01] and their variances met the assumption of homogeneity of variance [Fð1; 606Þ ¼ 0.93, p ¼ 0.76], we decided to perform the comparison using the nonparametric Mann-Whitney test [28].This test indicates that the scores obtained by students in the noncontext test (Mdn ¼ 7) did not differ significantly from the students in the mechanics test (Mdn ¼ 7), U ¼ 45011.5, z ¼ −0.56, p ¼ 0.578.So we can conclude that the differences between students' overall performances in both tests are not significant.Despite this fact, we did find significant differences in students' performances on some of the test items.In the next section we analyze these differences in detail.

V. STUDENT PERFORMANCE ON EACH TEST ITEM
In this section we address the second objective of the study, which is to understand the effect of context on students' understanding by comparing students' performance on each of the items of both tests that appear in the Supplemental Material [25].To compare students' performances and detect significant differences we use the chi-square test following the procedure described by Sheskin [29].First we used the chi-square test (p < 0.05) to determine whether there was a significant difference in the distribution of answers between the two problems.If there was, we then determined which specific option was significantly different using also the chi-square test.To avoid inflating the Type I error, we used the Bonferroni correction dividing the original p value (0.05) by the total number of comparisons (5, one for each multiple choice option) to obtain the new conservative p value of 0.01.Based on this analysis, we classified the items into the three different groups shown in Table III: (1) items in which the mechanics context helps students select the correct answer and there exist significant differences in the selection of incorrect answers, (2) items with no significant differences, and (3) items with significant differences in the selection of incorrect answers but not in the selection of the correct answer.
The types of items in the groups in Table III have interesting features.In the first group we clustered the two items in which we found that the mechanics context helps students select the correct answer; that is, we found a significant difference in the selection of the correct answer, and the proportion of students selecting the correct answer was higher in the mechanics context item.In these items we also found significant differences in the selection of incorrect answers.The interesting result is that these two items evaluate the vector operation of the dot product and in both of them the associated context is work.One item evaluates the interpretation of the dot product as a projection (item 2), and the other evaluates the calculation of this product using the equation AB cos θ (item 11).As we can see in the last two columns of Table III, the differences between the proportions of students selecting the correct answer in these two items is around 20%.
In the second group we clustered the six items for which we did not find significant differences in the students' performance.The associated contexts of these six items are force and velocity.Five items evaluate three vector properties: direction, magnitude, and components (see Tables II  and III) and the associated context is force.The sixth item evaluated graphical subtraction of vectors in 2D and the associated context is velocity.
In the third group we grouped the four items for which we found significant differences in the selection of incorrect answers but not in the correct answer.The items are graphical addition of vectors in 2D (item 1), graphic representation of a vector multiplied by a negative scalar (item 8), graphical subtraction of vectors in 1D (item 12), and calculation of the dot product of vectors written in unit vector notation (item 6).
It is worth noting that there are some interesting tendencies when we focus on the effect of the three different contexts in the mechanics test.We note that in the majority of items with the associated context of work, this context helps students in the selection of the correct answer (items of group 1).We also observe that in the items with the associated contexts of force and velocity (the items of group 2 and group 3), there are no significant differences in the selection of the correct answer.We will elaborate on these tendencies in the discussion section.Next, we investigate in detail the answers for each of the items of the tests.For each item we analyze (1) the type of difference noted between the tests in the selection of the correct answer, (2) the most common error on each of the tests, and (3) the differences in the selection of incorrect answers.
A. First group: Items in which the mechanics context helps students select the correct answer In this section we analyze the answers for the two items in the first group: items 2 and 4. In these items we found that the mechanics context helps students select the correct answer.We also found significant differences in the selection of incorrect answers.As mentioned before, these items evaluate the understanding of dot product.

Dot product
Interpretation of dot product.-Item 2 evaluates students' understanding of the interpretation of the dot product.Figure 1 shows item 2 of both tests.Results of these items are shown in Table IV.In the noncontext item, students have to choose the best geometric interpretation of the dot product of two vectors.In the mechanics context item, students have to choose the best geometric interpretation of the work done by a force F exerted on a box when it moves a displacement d, defining this work as the dot product F • d.It is important to note that these items have been analyzed in a previous investigation [16] (not with the details presented here).Note that the arrows in this and the following tables indicate which options have significantly different distributions determined by p < .01 on the chi-square test.
In this item we found a significant difference in the selection of the correct answer (option B).The percentage of correct answers is significantly higher in the work context item than in the noncontext item (52% vs 27%).So we can establish that for item 2 the work context helps students select the correct answer.To try to understand this tendency we included this item in the second test administration, asking students to write their reasoning.We found that students who selected the correct option in the work context item presented mainly two justifications.The less frequent justification was based specifically on the geometric interpretation of the dot product as a projection.An example that illustrates this type of reasoning is "My reasoning is based on the fact that we know that the dot product is a projection."The more frequent justification was based on the geometric interpretation of the work concept.An example that represents this type of reasoning is the following: "The box moves to the right, so it is logical to think that the force that causes this movement is the x component of the force vector.Therefore it is the projection of F onto d." From this we observed that the students used two different resources (different ways of thinking about a situation) [30] to consider the physical situation in the work context item: the geometric interpretation of the dot product and the geometric interpretation of work.By contrast, in the noncontext item students can only use the first resource.This difference among items seems to explain the significant difference in the correct answer percentages in both items.It is important to mention that students who use the second resource in the work context item do not necessarily know the geometric interpretation of the dot product as a projection.
In this item the most common errors are different for both contexts.In the noncontext item, the most common error was to interpret the dot product of two vectors as a vector between the two vectors (option C) and in the work context item, the most common error was to interpret work as a horizontal vector, that is, a vector in the direction of the displacement (option E).In examining these options more closely, we also found a significant difference in the selection of these two errors; we describe them below.
In this item we found significant differences in the selection of three incorrect options: C, A, and E. We found that the absence of context triggers the selection of option C (31% vs 11%).From the results of the second administration we found that the most common incorrect reasoning by students who chose this option was to relate the dot product to the addition of vectors as detected by Van Deventer [7].Moreover, option A refers to the dot product of two vectors as a magnitude of a vector between the two vectors.As shown in Table IV, the noncontext item triggers the selection of this option (30% vs 13%).In the second administration we found that the most common incorrect reasoning by students was to relate the scalar nature of the dot product with the magnitude of a vector.An example of this incorrect reasoning is "Dot product is a scalar, that's why its result is a magnitude and not a vector."It is interesting to note that the differences in the selection of these two options (A and C) were very high (approximately 20%) and seemed to be due to the abstract nature of the noncontext item and to the fact that students have more resources in the work item, as shown previously.
We also found in this item that the work context item triggers the selection of option E (18% vs 4%).In the work context this option refers to an interpretation of work as a vector in the direction of displacement.In the second administration we found that some students seemed to confuse the work done by the force with the component of the force that did the work.For example, a student justified his answer by stating that he only "takes into account the work in the x axis" showing this confusion between work and force.It is interesting to note that some previous studies [31] have also shown this confusion, which seems to trigger the selection of this option in the work context item.
Calculation of dot product as AB cos θ.-Item 4 evaluates students' understanding of the calculation of the dot product as AB cos θ.In the noncontext item students have to choose the calculation of the dot product of two vectors (A and B) that are at an angle θ as AB cos θ.On the other hand, in the mechanics context item, students have to choose the calculation of the work done by a force F exerted on a box at an angle θ when it moves a displacement d, defining this work as the dot product F • d.The results of item 4 on both tests are shown in Table V.
As shown in Table V, for item 4 of both tests we found a significant difference in the selection of the correct answer (option B).The proportion of students selecting the correct answer is significantly higher in the work context item than in the noncontext item (81% vs 60%); that is, the mechanics context contributed to students' selecting the correct answer.This difference seems to be also related to resource issues, as in the previous item.In the work context item, students have two possible resources: the definition of the dot product as AB cos θ, and the work formula Fd cos θ that they might have learned as the force in the direction of the displacement (F cos θ) multiplied by the displacement (d).However, in the noncontext item students only have the first resource.It is important to mention, as in the dot product interpretation item, that the students who know how to calculate the work using the equation Fd cos θ do not necessarily know that AB cos θ is the equation needed to calculate the dot product of two vectors.
In this item we found significant differences in the selection of three incorrect options (E, A, D).Note that the proportion of students selecting these options is higher in the noncontext item.The most common error in both contexts is the selection of A cos θB sin θ (option E, 13% in noncontext and 7% in the mechanics context).As mentioned in our previous investigation [14] students who chose this option in the noncontext item usually correctly calculated the x component of vector A as A cos θ, but incorrectly calculated the x component of vector B as B sin θ (instead of B), and then concluded that the dot product is the multiplication of these two expressions.The next most frequent error for which we found a significant difference is the selection of AB (option A), which is the direct multiplication of the magnitudes of the vectors.We found that 11% chose this option in the noncontext item and only 3% chose it in the work context item.The third error is AB sin θ (option D) which is the expression of the calculation of the magnitude of the cross product of two vectors (7% in noncontext and 2% in the work context).Note, finally, that the trigger of options E, A, and D seems to be due to the abstract nature of the item with no context.
B. Second group: Items with no significant differences in students' performance Items 5, 11, 3, 7, 10, and 9 are clustered in the second group.We did not find significant differences in students' performances on these items, which evaluate the vector concepts of direction, magnitude, components, and subtraction of vectors in 2D.

Direction of a vector
Item 5 evaluates students' understanding of the direction concept.In the noncontext item students have to choose the vector(s) that have the same direction as a vector at an angle of 45°; and in the mechanics-context item, students have to choose the force(s) exerted on different boxes that have the same direction a force exerted on a box at an angle of 45°.Table VI shows the results of item 5 on both tests.
In this item we did not find significant differences in students' performance.The proportion of students selecting the correct answer is slightly higher in the noncontext item (86% vs 81%), but this difference is not significant.The most common error in both versions of the test is option A. In this option, students include two vectors in their answer: a vector at 45°, which is correct and a vector at 56.3°, which is incorrect.In a previous investigation [12], we note that students choosing this option in the noncontext item justify their selection by mentioning that both vectors are pointing to the same "region" (northeast) as the reference vector or that both vectors have positive x and y components as the reference vector.The selection of this option is slightly higher in the mechanics context item (14% vs 9%); however, this difference is not statistically significant.

Magnitude of a vector
Item 11 evaluates students' understanding of the magnitude concept.In the context-free item students have to choose the calculation of the magnitude of the vector 2i þ 2j and in the mechanics context item students have to choose the calculation of the magnitude of the force 2i þ 2j (Newton).The results of item 11 of both tests are shown in Table VII.
We did not find significant differences in students' performance on this item.In the noncontext item, the proportion of students selecting the correct answer is slightly higher (78% vs 71%) but this difference is not statistically significant.The selection of the option C (4) is the most common error in both versions (9% in no context and 14% in mechanics context).In a previous study [2], we show that the majority of students choosing this option in the noncontext item make a calculation error when they applied the Pythagorean theorem.They incorrectly calculated 2 2 þ 2 2 as 16 (not 8), thereby obtaining the incorrect value of 4.

Components of a vector
Three items on both tests (items 3, 7, and 10) evaluate students' understanding of the component concept.
Representation of the y component of a vector.-Item3 evaluates students' understanding of the representation of the y component of a vector.In the noncontext item students have to choose the y component of a vector and in the mechanics context item students have to choose the y component of a force exerted on a box.The results of item 3 from both tests are shown in Table VIII.
In this item we did not find significant differences in students' performance.The proportion of students selecting  the correct answer is slightly higher in the noncontext but without a significant difference (80% vs 74%).The most common error in both tests is option A, that is, a y component with a longer magnitude than the correct one.The percentages for this option in both tests are very similar (12% in noncontext and 13% in force context).In previous investigations [14] we established that many students who incorrectly selected this option in the noncontext item believed that the magnitude of the y component of a vector must have the same value as the vector.
Representation of the x component of a vector.-Item7 evaluates students' understanding of the representation of the x component of a vector.This item is similar to item 3 (previously analyzed).For the noncontext item students have to choose the x component of a vector and for the mechanics context item students have to choose the x component of a force exerted on a box.The results of item 7 on both tests are shown in Table IX.
We did not find significant differences for this item in students' performance.The proportion of students selecting the correct answer is very similar in both contexts (78% in the free context and 77% in force context).The most common error in both tests for this item is the selection of option A, that is, an x component with a longer magnitude than the correct one.In previous investigations [14] we showed that the students' reasoning for selecting these options in the noncontext item was very similar to those established for item 3: many students incorrectly believed that the magnitude of the x component of a vector must have the same value as the vector.
Calculation of the x component of a vector.-Item10 evaluates students' understanding of the calculation of the x component of a vector when the angle (ϕ) is measured from the y axis.In the noncontext item students have to calculate the x component of a vector and in the mechanics context item students have to calculate the x component of a force exerted on a box.Table X shows the results of item 10 from both tests.
We did not find significant differences in this item with regard to students' performance.The correct answer in this item is the vector magnitude multiplied by sin ϕ, since ϕ is measured from the y axis.As shown in Table X, the proportion of students selecting the correct answer is slightly higher in the noncontext but without a significant difference (70% vs 64%).The most common error in both tests is choosing the expression that multiplies the vector magnitude by cos ϕ.Van Deventer [7] noted that students tend to memorize this expression in the noncontext item to calculate the x component of a vector; however, this expression is only valid when the angle given is measured from the x axis.

Subtraction of vectors in 2D
Item 9 evaluates students' understanding of the subtraction of vectors in 2D.In the noncontext item, students have to choose the representation of a subtraction vector of two vectors.On the other hand, the mechanics context shows the velocities in two instants (v 1 and v 2 ) of a car following a path and students have to choose the representation of the change of velocity vector, that is, the vector difference v 2 − v 1 .Table XI shows the results of item 9.
We did not find significant differences in students' performance in this item.The proportion of students selecting the correct answer is higher in the noncontext but with no significant difference (41% vs 33%).Option D is the most common error in both contexts.In this item students have to choose the subtraction vector A − B of vectors A ¼ −3i þ 3j and B ¼ −2i − 2j.Option D represents an incorrect answer: vector −1i þ 1j.In the analysis of the second administration we found, as in our previous study [1], two incorrect procedures.In the first procedure, which is the most common; students "directly" subtract the magnitude of vector B from the magnitude of the vector A; they think that the direction of the subtraction vector is in the direction of vector A because it has the greater magnitude.In the analysis of students' reasoning, we found the following explanation in the noncontext problem: "Vector A is greater that vector B so when we subtract the magnitude of vector B, it remains a smaller vector in the direction of vector A." It is interesting to note that some of the students following this incorrect procedure also incorrectly assigned magnitudes to vectors A and B. They mistakenly thought that the magnitude of vector A is 3, the magnitude of vector B is 2, and when they performed the subtraction they calculated the vector difference as a vector with 1 unit.The next reasoning illustrates this error: In the second procedure students subtracted the two vectors using components, and incorrectly added the y components of the two vectors.
C. Third group: Items with significant differences in the selection of incorrect answers but not in the selection of the correct answer In the third group we analyze the two items for which we found differences in the selection of incorrect answers but not in the selection of the correct answer: graphical addition of vectors in 2D (item 1), negative scalar multiplication of a vector (item 8), subtraction of vectors in 1D (item 12), and calculation of the dot product of vectors written in unit vector notation (item 6).

Addition of vectors in 2D
Item 1 evaluates students' understanding of the addition of vectors in 2D.In the noncontext item students have to choose the representation of the vector sum of two vectors, and in the mechanics context students have to choose the representation of the net force of two forces acting on a box.Table XII shows the results of item 1.
As shown in Table XII, we did not find a significant difference in the selection of the correct answer.In the noncontext item, the proportion of students selecting the correct answer is 56% and in the mechanics context item it is 50%.The most common errors in this item are different for both contexts.In the noncontext item, the most common error is a tip-to-tip (option D) and in the force context item the most common error is a general bisector vector (option A).A bisector vector is a vector that goes between the two vectors but lacks the precision to be considered correct [7].
In this item we found significant differences in the selection of options A, B, and D. Option A represents a general bisector vector and option B represents a horizontal bisector.We found, similar to what we found in a previous study [13], that both options are triggered in the force context item: 27% vs 9% in option A and 15% vs 8% in option C. The trigger for choosing a bisector vector seems to be the fact that students usually draw "approximate" net force vectors in the free body diagrams in their mechanics course.Option D represents a tip-to-tip error first reported by Knight [3].Again, similar to what we found in a previous study [13], we noted that this error is triggered in the noncontext item (24% vs 7%).It seems that in the absence of context many students cannot visualize a vector sum in the same way they can visualize a net force vector, and this can lead them to choose the tip-to-tip vector error, which is a vector with a direction very different from the correct one.Finally, it is interesting to note that the selection of incorrect answers with significant differences cancel one another out, since the proportion of students selecting the correct answer is the same in both contexts.

Negative scalar multiplication of a vector
Item 8 evaluates students' understanding of the negative scalar multiplication of a vector.The noncontext item shows a vector A and students have to choose the vector −3A, and the mechanics context shows a box moving at a velocity vector v and students have to choose the representation of the box moving at a velocity −3v.Note that the noncontext problem was analyzed by us in a previous investigation [15] (not with the details presented here).Table XIII shows the results of item 8 on both tests.
As shown in Table XIII, we did not find a significant difference in the selection of the correct answer.In the noncontext item, the proportion of students selecting the correct answer is 69%, and in the velocity context item, the proportion of students is 61%.In this item the most common errors are different for each test: option A in the noncontext item and option E in the velocity context item.We found significant differences in the selection of option E (17% vs 5%) and option A (12% vs 6%).The velocity context triggers the selection of option E, which refers to a vector with incorrect magnitude.In the analysis of students' reasoning in the second administration, we found that the velocity context triggers an incorrect belief in which students relate the "3" value of the expression −3v with a "vector with a magnitude of 3" (not a three times magnitude).In contrast, the noncontext prompts the selection of option A, which refers to a "perpendicular vector."In our analysis of the second administration, we found that the absence of context triggers an incorrect assumption in which students relate the negative sign of the expression −3A to a vector pointing "to the right," "to the positive side," or "to the opposite side."Therefore, in this item we found two different tendencies in the selection of incorrect answers.In the first, the velocity context item triggers the selection of a vector with incorrect magnitude; and, in the second, the noncontext item triggers the selection of a vector with incorrect direction.

Subtraction of vectors in 1D
Item 12 evaluates students' understanding of the subtraction of vectors in 1D.In the noncontext case, students have to choose the representation of a subtraction vector of two vectors.(In the noncontext item, vector A is a vector with three units to the left, vector B is a vector with 5 units to the right and students have to choose the vector difference A − B, that is, a vector with eight units to the left.)On the other hand, the mechanics context shows the initial and final velocity vector (v i and v f ) of a car colliding into a wall and students have to choose the representation of the change of velocity vector that is the vector difference v f − v i .Table XIV shows the results of item 12.
As shown in Table XIV, we did not find a significant difference in the selection of the correct answer.In the noncontext item, the proportion of students selecting the correct answer is 46%, and in the velocity context item, the proportion of students is 36%.The most common error in both items is option B, which represents the vector sum of both vectors.We did not find a significant difference for this option.In the analysis of students' reasoning in the second administration, we found three frequent incorrect procedures.A common incorrect procedure is to use a graphical addition procedure instead of a subtraction procedure (as detected by Van Deventer [7]).The other two incorrect procedures occurred when students tried to solve this problem using numbers with positive or negative signs according to the direction of the vectors.An incorrect procedure is "A − B ¼ −3 þ 5 ¼ 2," in which students make an error in the sign of the number 5, since it should be negative.The other incorrect procedure is "A − B ¼ 3 − 5 ¼ 2," in which students consider that the two vectors are in the same direction.
In this item we found significant differences in the selection of options C and A. Option C represents a vector with the same magnitude as the vector sum of the two vectors but opposite direction (hence the name "opposite vector sum").The proportion of students selecting this option is much higher in the velocity context item (18% vs 2%).Note that in the velocity context students have to choose the representation of the change of velocity vector; that is, the vector difference v f − v i .In the analysis of students' reasoning in the second administration, we found that the most common incorrect procedure for choosing option C in the velocity context usually has two parts.Students calculate the magnitude of the change of velocity vector using numbers with positive or negative signs, following an incorrect procedure similar to what they use to calculate the vector sum (option B) shown before (i.e., "A − B ¼ −3 þ 5 ¼ 2").Then they state that the direction of this change of velocity vector "has to be opposite the initial velocity and in the same direction as the final velocity."This is in fact the correct direction of this vector and a possible correct reasoning for this problem.This type of reasoning allows us to understand the great difference between the selections of this option (18% vs 2%).We note that the concept used in the velocity context "change of velocity vector" (which is not used in the noncontext item) seems to trigger the selection of this option in the velocity context.Van Deventer [7] administered a similar problem and also found a difference in the selection of an option with the same characteristics as option C for our items.He hypothesizes that this difference may be due to a velocity and position confusion.However, according to our results this trigger seems to be more likely due to the use of the "change of velocity" concept as mentioned previously.
The other significant difference occurs in the selection of option A, which represents a vector sum of both vectors with incorrect magnitude.The velocity context item triggers the selection of this option (4% vs 0%).In the figure of item 12 with velocity context, one can observe that the figure of the cars may result in students getting confused of the magnitudes of both vectors, which may trigger this selection.Although there is a significant difference in the selection, note that both proportions of this selection are very small, less than 5%.4. Calculation of the dot product of vectors written in unit vector notation Item 6 evaluates students' understanding of the calculation of the dot product of vectors written in unit vector notation.In the noncontext item, students have to choose the calculation of the dot product A • B of vectors A ¼ 1i þ 3j and B ¼ 5i.In the mechanics context item, students have to choose the correct calculation of the work (in Joules) done by a force F ¼ 5i þ 3j (N) exerted on a box when it moves a displacement d ¼ 5i (m), defining this work as the dot product F • d.The results of item 6 are shown in Table XV.
In this item we did not find a significant difference in the selection of the correct answer.In both contexts the proportion of students selecting the correct answer is 30%.The most common error in both items is option C (5i þ 3j).In this option we did not find a significant difference.In the analysis of the second administration we found, as in our previous study [1], that students who choose this option usually calculate the product of x components 1i (from vector A) and 5i (from vector B) as 5i and then add the vector 3j (from vector B) to the result.
In this item we found a significant difference in the selection of option E (5i), which represents a vector with the same direction as the displacement vector.The proportion of students selecting this option was higher in the work context item (21% vs 12%).Students who made this error followed the same procedure as described above for the error 5i þ 3j, but did not add the vector 3j.In the analysis of students' reasoning in the second administration, we found that the work context triggers the same incorrect reasoning as the one established in the interpretation of the dot product item.As mentioned before, students confuse the work done by the force with the component of the force that did the work.This interpretation seems to be what prompts the selection of option 5i in the work context item.

VI. DISCUSSION
In this section we discuss the main findings of this study, presenting instructional recommendations, and comparing these findings with the results of previous research.

A. Students' overall performance on both tests
We found no significant differences in students' overall performances on both tests (objective 1).The medians in both tests are 7 (out of 12) which means that students who are on the median in both tests have difficulty correctly answering 5 items.The Mann-Whitney test indicates that the scores obtained in both tests did not differ significantly.With the analysis of students' performance on all the items of both tests (objective 2), we can describe this general result to a greater extent.As shown before, we found that in the two items of the first group, a higher proportion of students choose the correct answer (with statistically significant differences) on the TUV-mechanics test compared to that on the noncontext test.On the other hand, if we see the results of other items (items 8, 9, and 12 in Table III), we observe that a slightly higher proportion of students choose the correct answer (with no statistically significant differences, but with differences between 8% and 10%) on the TUV-noncontext test compared to that on the mechanics test.These results seem to cancel each other out in such a way that students' overall performances on both tests are similar.Observe that these three items (8, 9, and 12) are those with the associated mechanics context of velocity and will be discussed in the overview analysis below.
An interesting analysis of students' overall performance is the identification of the most difficult items for the students.Table III of this article presents the correct percentages in all of the items of both tests.We observed that five items have correct percentages of less than 60% in both types of tests, showing that they are the most difficult items.These five items are geometric interpretation of dot product as a projection (item 2), graphical subtraction of vectors in 1D and also in 2D (items 9 and 12), graphical addition of vectors (item 1), and calculation of the dot product of two vectors in unit-vector notation (item 6).Therefore, our first general recommendation for the instruction of vector concepts in a mechanics course is to specifically focus on these concepts in both contexts.

B. Students' performance on all of the items of both tests
In this study we also compared students' performance on all of the items of both tests (objective 2).All of the items of both tests were clustered in three different groups, according to the differences in students' performances (Table III).Next we discuss the main findings in each of these groups and we compare them with the results of previous studies, mainly with the thesis of Van Deventer [7].Regarding the latter, it is important to mention that Van Deventer's work focuses on determining if a context effect exists in the selection of the correct answer, but does not examine whether the physical context or the absence of context helps the students in the selection of the correct answer.In this work we do this analysis and that allows us to establish specific instructional recommendations.At the end of this section we present an overview analysis of the context effect in all of the items.

First group: Items in which the mechanics context helps students select the correct answer
The two items in this group evaluate the interpretation of the dot product (item 2) and the calculation of the dot product as AB cos θ (item 4).These items have two important characteristics that should be noted: they evaluate the understanding of the dot product, and the associated mechanics context in both items is work.
In the items of this group, we found that the work context helps students select the correct answer.As established before, the main reason for this tendency is that students have more resources in the work context items of this group.This is a very important result and we recommend that teachers consider it in their instruction.In physics courses, instructors usually teach the mathematical concept of dot product first, and then they apply this concept to the work context.Given that the context of work helps students in the interpretation and calculation of the dot product using the equation AB cos θ, it seems advisable for instructors to follow the usual process of first teaching the dot product and then the work concept, but at the end to return to teaching the dot product.In this way students can strengthen their grasp of the dot product in the mathematical context having available the "extra" resource that they learned in the work context.
In analyzing students' performance of these items, we have described the most frequent errors in each context and identified the incorrect options whose selections were triggered by each of the contexts.We recommend that instructors consider this information.Finally, it is important to mention that Van Deventer [7] does not include items that specifically evaluate these concepts.

Second group: Items with no significant differences in students' performance
In this group we clustered six items.Five items evaluate three vector properties: direction, magnitude, and components (see Tables II and III).It is interesting to note that these are the only items that evaluate properties in the test (the other items are vector operations) and that in these five items the associated context is force.The sixth item evaluates graphical subtraction of vectors in 2D and the associated context is velocity.
In these items we found no significant differences in the selection of the correct and incorrect options between both contexts.The students had a similar understanding of these concepts in both the mathematical and associated contexts.For these items we also described the most frequent error in all of these items that turned out to be the same, regardless of the context.Our general recommendation to teachers who wish to increase their students' understanding of these concepts is to take into account the information about students' difficulties we present here.Now we compare our results for these items with the results obtained by Van Deventer [7].First, note that his study does not evaluate the direction concept.Moreover, Van Deventer included a similar magnitude problem in which students have to calculate the magnitude of a vector as a function of their components, and also found no significant differences in the selection of the correct answer.In addition, Van Deventer included items that evaluated the representation of components, and found, in contrast to our results, significant differences in the selection of the correct answer in both problems.However, Van Deventer established that strict isomorphism of these items was broken and mentioned that this may have contributed to the differences that were found.In our research we designed items with strict isomorphism and found no significant differences in the selection of the correct answer.In the fifth item of the first group, calculation of the x component of a vector, the results of both studies show no significant differences in the selection of the correct answer for this type of problem.Finally, in the sixth item, subtraction of vectors in 2D, associated context of velocity, Van Deventer found a significant difference in the selection of the correct answer.We did not find it.This discrepancy in the results may be due to the fact that the differences in this item are near the critical value that splits the acceptance and rejection regions, so the detection of these differences will depend on how conservative the statistical methods used are.This issue is discussed below.
3. Third group: Items with no significant differences in the selection of the correct answer but with significant differences in the selection of incorrect answers Four items are in this group: addition of vectors (in which the associated mechanics context is force), negative scalar multiplication and subtraction of vectors in 1D (in which the associated context is velocity), and calculation of the dot product of two vectors written in unit-vector notations (in which the associated context is work).
In these items we did not find a significant difference in the selection of the correct answer among students finishing an introductory mechanics course.Students had a similar understanding of these operations in both the mathematical and associated contexts.However, in these items we found significant differences in the selection of incorrect answers.We recommend that instructors take into account the errors that are triggered by each context and the most frequent error in each context.Now we compare our results for these items with the results obtained by Van Deventer [7].First, note that in that study the negative scalar multiplication was not evaluated.Moreover, in that study a similar addition and calculation of the dot product in unit-vector notation were included with also no significant differences in the selection of the correct answer.
Regarding the subtraction of vectors in 1D that has the associated context of velocity, Van Deventer [7] found a significant difference in the selection of the correct answer, while in this study we did not find this difference in agreement to Wang and Sayre's conclusion [8].As established previously, these discrepancies seem to be due to the fact that the differences are close to the critical value.This issue is discussed below.
After the analyses, we can now establish why the result of this dot product item, which evaluates the calculation of this product with vectors written in unit-vector notation, is not the same as the other two dot product items from the first group.The item in this third group is more procedural than, and not as conceptual as, the other two and that seems to explain the difference.

Overview analysis of the context effect in all of the items
Analyzing the context effect we observe that the work context helps students in the selection of the correct answer in all of the items that evaluated the conceptual aspects of the dot product (interpretation of the dot product and the calculation of the dot product as AB cos θ).As mentioned previously, the item of calculation of the dot product of vectors written in unit vector notation that has the associated context of work (item 6 of group 3) is a more procedural problem and does not fit in this category, since it seems that the similar correct percentages in both contexts of this item are due mainly to its procedural character.We also observe that in the items with the associated contexts of force and velocity (the items of group 2 and group 3) there are no significant differences in the selection of the correct answer.
Next, we explain the different effects of the contexts.As mentioned before, the work context helps the students in the conceptual problems, since in this context students have more resources; i.e., work allows students to have an "extra" physical interpretation that is "close" to them.In the item that asks for the interpretation of work defined as F • d, we realize that students can perform the interpretation using the physical interpretation (force in the direction of the displacement multiplied by the displacement) or the geometric interpretation of the dot product as a projection.In contrast, in the item that asks for the interpretation of the dot product of two vectors with no context, students only have the latter interpretation.The same happens in the item that asks students for the calculation of work as Fd cos θ and in the item that asks for the calculation of dot product in a noncontext problem as AB cos θ.We think that this extra physical interpretation is the reason why the work context is helpful to the student.
But what happens in items with the context of force and velocity?Do these contexts also provide this extra physical interpretation?If we analyze the items with these contexts, we can see that in these cases the physical mathematical interpretations are so closely related that students do not receive any extra physical interpretation.Consider, for example, the addition of vectors in 2D in the context of force; to answer this question correctly, students must perform a physical interpretation that by definition includes the vector interpretation.The same happens with other items.We think that not having this extra interpretation explains why the context does not help students in the force and velocity cases.
Once we have explained the effect of the contexts, we consider that is important to focus on three items (items 8, 9, and 12), those with the associated mechanics context of velocity.These items evaluate two vector operations: negative scalar multiplication and subtraction of vectors (in 1D and 2D).Note that these three items all used the negative of a vector concept.As mentioned before, in these items we did not find significant differences on the selection of the correct answer by statistical tests using a corrected p value of 0.01 based on the Bonferroni correction to avoid inflating the type I error.We also noted that in two of these three items (subtraction of vectors in 1D and 2D) there were discrepancies comparing to results from Van Deventer [7] and that we believed that it could be due to the fact that the differences in these items were near the critical value.Taking into account these discrepancies, we consider it important to discuss focusing on the adjusted p value used in the present study.
Sheskin [29] establishes that some researchers consider that this correction is too severe (since it substantially reduces the power associated with each comparison) and that in the final analysis one must decide what per comparison p value results on a fair balance in terms of the likelihood of committing the type I error and the power associated with a comparison.So the question that arises is: What happens to the results if we decide to do an analysis that is not too severe and is conducted with a p value of 0.05 (not 0.01)?If we decide to do so, we also find significant differences in the selection of the correct answer in the three items with the associated mechanics contexts of velocity.In all of these three items the proportion of students choosing the correct answer is higher in the noncontext items, so in these three items students have in some degree more difficulties to correctly answer when presented the velocity context.We consider that this issue is interesting for several reasons.First, because this stronger difficulty in the mechanics context (compared to noncontext) is only found in these three items of the entire test; second, because some other studies (as Van Deventer [7]) have detected the same result with items of subtraction of vectors in 1D and 2D; and finally, because of instructional purposes.
The question that arises is: Why do students have more difficulties in the velocity context?Maybe the best way to answer this question is to analyze all together the three errors that are triggered by the three items with this context.We realize that these errors have something in common, that in their reasoning process the students stop considering the velocities as vectors and treat the magnitude of these velocities as scalars.For example, in the item that asks for the representation of a box moving at −3v, students treat the magnitude of the velocity as a scalar, assigning it a value of 3 (option E).Moreover, in the subtraction of velocities of 1D and 2D, students who choose the incorrect answers "opposite vector sum" (option C) and "vector −1i þ 1j" (option D), respectively, treat the velocities as magnitudes.In the first incorrect answer "opposite vector sum," students make sign mistakes, and in the second incorrect answer "vector −1i þ 1j" they make conceptual mistakes, since they "directly" subtract the magnitudes of the vectors as scalars.Finally, we raise the question: Why does this happen?We believe the answer may be related to the use of the velocity context in the introductory mechanics course.If we consider a traditional textbook [26] or a traditional curriculum, we realize that in the one-dimensional velocity context students mainly solve problems in which they treat the velocities as scalars with signs and not as vectors (consider, for example, the use of the equation v f ¼ v o þ at).This also happens in two-dimensional situations.For projectile motion, students usually analyze the horizontal and vertical motions separately, also treating the components of the velocities as scalars.In addition, for uniform circular motion, students mainly use the constant speed of the particle (i.e., a R ¼ v 2 =R), again treating the velocity as a scalar.We believe that because of this frequent use of the velocity as a scalar, students have difficulties with problems in which it is necessary to treat the velocities as vectors and commit errors with procedures in which the velocities are treated precisely as scalars.Finally, it is interesting to note that this does not happen in the context of force and work in a traditional textbook [26] or a traditional curriculum.If we look at the use of these contexts, we observe that students typically treat the forces associated with both contexts as vectors, usually making free body diagrams.

VI. CONCLUSION
In this article we found no significant difference between the medians obtained in the noncontext and mechanics context tests; however, we did find significant differences in some items.We observed that the understanding of vector concepts by students who were finishing a calculus-based mechanics course depended on the type of context.
In the items with the associated context of work that evaluated the interpretation of the dot product and its calculation as AB cos θ, we found that the work context helps students select the correct answer.We observed that this occurs because the work context affords students an extra physical interpretation that is close to them.Because of this benefit, it seems advisable for instructors to follow the usual process of first teaching the dot product and then the work concept, but at the end to return to teaching the dot product in order to maximize the value of this extra resource.
In the items with the associated context of force, we did not find significant differences in the selection of the correct answer.These problems involve vector properties (direction, magnitude, and components) and vector operations (addition).We noted that this occurs because in this context the physical interpretation and the mathematical are so closely related that students do not benefit from any additional physical interpretation.
In the items with the associated context of velocity that evaluate two vector operations (negative scalar multiplication and subtraction of vectors in 1D and 2D) which both of them use the concept of negative of a vector, we detected an interesting tendency.Using a less severe statistical procedure we found, as other studies [7], that in these items students had more difficulties selecting the correct answer in the velocity context items.We noted that this occurs because the velocity context causes the students to incorrectly treat the velocities as scalars.Because of this, it seems advisable for instructors to take into consideration the fact that students have a limited ability to solve this specific kind of velocity vector problem.
Furthermore, in problems that have a more procedural character (as in the calculation of the dot product of two vectors written in unit vector notation) we did not find differences in the selection of the correct answer.We believe that this tendency was probably due to the procedural character of the items and not to the context effect.
Finally, in this article we described the most frequent errors for all of the items in each context and identified the incorrect options that were triggered (if that was the case) by each of the contexts.We recommend that instructors who wish to increase their students' understanding of these concepts in both contexts take this information into account when planning their instructional methodologies.
One limitation of the present study is that for the majority of vector concepts we use only one problem to evaluate students' understanding.However, the similar tendencies that we found in all of the items with the same context seem to confirm what we have stated for each of the vector concepts and for each of the contexts.Also our results were consistent with what others have found in similar studies (when available).In a future study based mainly on interviews, we will try to investigate in more detail the general tendencies established in this article and their likely causes.Finally, it is important to mention that future studies could consider the instructional recommendations established in this article to design new instructional material that fosters better students' understanding of vector concepts in noncontext and mechanics context problems.

FIG 1 .
FIG 1. Item 2 of the TUV-12 items no context and the TUV-12 items mechanics that evaluate the geometric interpretation of the dot product as a projection in both contexts used in the first administration.Note that in the second administration we also included at the end of these items the question "Explain your reasoning."

TABLE II .
Description of both tests used in this study.

TABLE III .
Classification of items according to the differences in students' performance.Note that to determine which specific option was significantly different we use the chi-square test with a p value of 0.01 (adjusted by the Bonferroni correction).

TABLE IV .
Results of item 2: Students have to interpret the dot product of two vectors or work defined as a dot product as a projection.The arrows indicate which options have significantly different distributions determined by p < .01 on chi-square test.

TABLE V .
Results of item 4: Students have to calculate the dot product of two vectors as AB cos θ or the work done by a force in a displacement as Fd cos θ.Arrows indicate significant differences.

TABLE VII .
Results of item 11: Students have to calculate the magnitude of the vector or force 2i þ 2j.

TABLE VIII .
Results of item 3: Students have to choose the y component of a vector or force.

TABLE VI .
Results of item 5: Students have to choose vector(s) or force(s) with the same direction as a vector or force at an angle of 45°.

TABLE IX .
Results of item 7: Students have to choose the x component of a vector or force.

TABLE X .
Results of item 10: Students have to calculate the x component of a vector or force (angle ϕ measured from y axis).

TABLE XII .
Results of item 1: Students have to choose the vector or force addition in 2D.Arrows indicate significant differences.

TABLE XIV .
Results of item 12: Students have to choose the representation of the vector difference or change of velocity vector in 1D.Arrows indicate significant differences.

TABLE XV .
Results of item 6: Students have to calculate the dot product of two vectors or force and displacement written in unit vector notation ½ð1i þ 3jÞ • ð5iÞ.