Transforming common-sense beliefs into Newtonian thinking through Just-In-Time Teaching

To determine whether teaching an introductory physics course with a traditional lecture style or with Just-in-Time teaching a student-centered, interactive-engagement style will help students to better understand Newtonian concepts, such as Newton’s Third Law, 222 students in introductory physics courses taught by traditional lecture styles and Just-in-Time teaching at North Georgia College & State University over the span of five semesters were examined using the Force Concept Inventory as a pretest and a post-test. Overall, the gains favor the Just-in-Time teaching method with a 37.6% 2.0% gain compared to the 17.9% 2.5% seen in traditional lecture classes. When analyzing only those gains pertaining to the Newton’s Third Law questions, the results again favor the Just-in-Time teaching method with a gain of 50.8% 4.1% while the traditional lecture classes only saw a gain of 6.6% 5.2%. We also employed a new method of analysis which was a BIT Coding method created to quickly identify students’ understanding of Newton’s Third Law questions. This study shows that students in courses that are taught using the Just-in-Time teaching strategy better understand Newton’s Third Law after instruction than do students in traditional lecture courses.


I. INTRODUCTION
When students embark on their first university physics class, they usually do not consider the laws of motion from a Newtonian point of view, but instead use common sense ͑CS͒ thinking, as described by Halloun and Hestenes in 1985 ͓1͔, or, in other words, they think with an Aristotelian point of view. It is this Aristotelian thinking that leads students to misinterpret and misunderstand many of the concepts discussed in most introductory physics courses. Halloun and Hestenes state that, in a traditional lecture course "the students operate with a seriously defective conceptual vocabulary, which implies that they continually misunderstand the material presented" ͓1͔. In a letter to The Physics Teacher, Huffman and Heller agreed with Halloun and Hestenes that students only have a loosely organized, incoherent understanding of forces ͓2͔. In an introductory physics course, students' views about force and motion are not classified in any logical manner and are typically context dependent. Halloun and Hestenes' recommended solution to this conceptual misunderstanding was sensitive student-centered instruction, which has more recently been termed active learning or interactive engagement ͑IE͒.
To better understand students' prior knowledge and pre-Newtonian misconceptions they might have before taking an introductory physics course, Hestenes, Wells, and Swackhamer developed the Force Concept Inventory ͑FCI͒ ͓3͔, a multiple-choice exam that makes students choose between Aristotelian and Newtonian theory when answering each question. Hestenes et al. describe the FCI not as a "test of intelligence" but as a "probe of belief systems," and suggest that a score of 60% implies the student has reached the entry threshold of understanding Newtonian concepts, while a score of 85% is the Newtonian Mastery threshold and students with this score are established Newtonian thinkers ͓4͔. Since its introduction, the FCI has been used in physics education research ͑PER͒ to probe student understanding of force concepts and the data collected with the instrument has been analyzed in many different ways. One of the most common methods of pretest to post-test assessment is the g factor, or normalized gain, g = Post-test score − Pretest score Maximum score − Pretest score , ͑1͒ which was proposed by Ghery ͓5͔, and made ubiquitous in the PER community by Hake ͓6͔. This measure indicates a student's knowledge-state transition from preinstruction to postinstruction; in other words, the normalized gain quantifies student learning. The normalized gain has been measured in hundreds of classes and for thousands of students worldwide with extremely consistent results. Hake showed that, for more than 2000 introductory physics students in traditional lecture courses, the average normalized gain on the FCI was 23% Ϯ 4%. Whereas, for more than 3000 introductory physics students in IE courses, the normalized gain on the FCI was 48% Ϯ 14% ͓6͔, which is quantitative evidence that the student-centered, interactive-engagement teaching method promotes a greater conceptual understanding of Newton's laws of motion than the traditional, passivestudent lecturing method. There exist a variety of teaching and learning methods in introductory physics that could be categorized as student centered, which rely on interactive engagement for active learning. Examples of these techniques include the University of Washington's Tutorials in Introductory Physics ͓7͔, Sokoloff and Thornton's Interactive Lecture Demonstrations ͓8͔, the University of Maryland's Activity-Based Tutorials ͓9͔,Physlets ͓10͔,Peer Instruction ͓11͔, In this study, student learning and understanding of Newtonian concepts were measured for two independent groups: introductory physics students taught by traditional lecture techniques and introductory physics students taught with the Just-in-Time Teaching ͑JiTT͒ strategy. It will be shown that the students in the JiTT group developed a better conceptual understanding of Newton's laws than those students taught in the more traditional manner. Examples of Newton's Third Law ͑N3͒ concepts will be used to show that the JiTT teaching strategy promotes conceptual understanding and helps to eradicate the Aristotelian misunderstandings that often hold students back from becoming established Newtonian thinkers.

Just-in-Time Teaching
Just-in-Time Teaching is a pedagogical strategy that employs the Internet to develop and utilize a feedback loop between students and instructors that exists both in class and out of class. The JiTT method was developed by Novak, Patterson, Gavrin, and Christian in 1999 ͓13͔ and has been adopted by more than 300 faculty at more than 100 institutions worldwide. Although there does exist a book published on JiTT, numerous JiTT workshops, and great numbers of enthusiastic JiTT supporters, the literature is lacking rigorous peer-reviewed data to evaluate the efficacy of the teaching method, which is one of the objectives of this paper.
The JiTT method involves a three-step process: ͑1͒ a preinstruction reading assignment with concept-based questions ͑ie., the WarmUp exercise ͓13͔͒, ͑2͒ an in-class discussion of the conceptual questions, and ͑3͒ a group activity involving the concepts discussed in the reading and lecture. Students are required to answer the preinstruction WarmUp questions and submit their responses via the Internet a few hours before lecture begins. The instructor reviews the student responses to the questions and designs the lecture structure and activities according to the students' ͑mis͒understanding of the concepts.
During the first step of the JiTT process, students read an assigned portion of the text and then answer conceptual questions related to the reading. This WarmUp gives the students a brief introduction to the topic, which provides a general idea of what the day's lecture will cover. The students are not expected to comprehend everything in the reading, but they are encouraged to think critically about the text. The conceptual questions impel the students to reflect on the text, relate the concepts to everyday life, and write about the concepts in a conversational manner. The WarmUp questions must be answered a few hours before class meets, which ensures that the day's concepts are fresh on the students' minds. In the JiTT classes discussed in this paper, the students were graded on the WarmUp questions according to the effort they put forth in their responses and not on the accuracy of their answers. Their WarmUp exercises counted for 10% of their final grade. During the few hours before class, the instructor reviews the student responses and determines the state of the class' understanding of the material. The instructor also chooses some of the student responses to present to the class to establish an interesting and informative class discussion. Additionally, through the course webpage or email, the instructor can provide feedback to students faster than how feedback is usually provided in a traditional lecture course. For classes of approximately 35 students, the instructor spent roughly 30-45 min reviewing the WarmUp exercise responses before each class.
The second step in the JiTT process entails the class discussion of the students' responses to the WarmUp questions.
The instructor presents the WarmUp question͑s͒ to the class, along with some carefully chosen student responses that will spark class discussion. During this discussion, many of the most common misconceptions about the material are argued and shown to be invalid, while the Newtonian theory of the concept is verified. Students are often clearly excited when their WarmUp response is included in the discussion, so not only does this step in the JiTT process motivate students to participate, but this process also forces the students to discuss Newtonian concepts using everyday, conversational English rather than parroting memorized text.
Finally, the Newtonian concepts discussed in the first two JiTT steps are applied in a group activity designed to relate the concepts to everyday life. This activity, which could involve a demonstration, simulation, word problem, etc., enables the students to be active learners and to learn by doing rather than just listening in a passive traditional lecture format, which research ͓6͔ has shown does not promote student learning. The students get instant feedback from the instructor and also from their peers during the group activity.
The JiTT method promotes active learning and conceptual understanding by first introducing the concepts to students through the text, then compelling the students to critically reflect on the material and write about the concepts in their own words. The class discussion that follows helps the students to eradicate their previous Aristotelian beliefs and understand the Newtonian point of view. To solidify their understanding, the students apply the concepts to a group problem in which they see how the concept is useful in real life, outside of the classroom.

II. RESEARCH INSTRUMENTS AND METHODS
To compare the effects of different teaching strategies and to measure student learning and understanding of Newtonian concepts, 222 students in Introductory Physics courses ͑both algebra-and calculus-based͒ at North Georgia College & State University, were studied over the course of five semesters. These students were administered the FCI exam as a pretest, before any formal instruction was given, and then again as a post-test, after the Newtonian laws of motion were covered in their courses. The students were rewarded extra credit points contingent upon their performance on the FCI exam in order to motivate the students to take the exam seriously. About one third of the students were in calculusbased introductory physics and the other two thirds were in the algebra-based course, with each of the classes being comprised of approximately 25-35 students. Most of the students were science majors ͑physics, chemistry, biology͒ and were taking the course because it was required for their major. Three different professors of varying tenure, ϳ20, ϳ10, and ϳ3 years, taught the introductory physics courses independently. The FCI pre-and post-test data were divided into two groups: JiTT, which contained data from students in both algebra-and calculus-based courses who were taught with the JiTT teaching strategy, and Non-JiTT, which contained data from students in both algebra-and calculus-based courses taught with more traditional lecturing methods. Only one of the professors, an assistant professor with three years experience, used the JiTT strategy while teaching the introductory physics courses.
The FCI pre-and post-test data were analyzed using three different methods, the normalized gain, a BIT Coding of four FCI questions on Newton's Third Law ͑N3͒, and a CS-Distractor investigation of students' responses on the four N3 questions. The normalized gain was calculated using the students' pretest and post-test scores on all 29 FCI questions to quantify student learning of Newton's laws of motion. In addition, the normalized gain was also calculated using the students' pretest and post-test scores on the four N3 questions. This "concept gain" was used to measure student learning of Newton's Third Law concepts. To probe deeper into students' misconceptions about Newton's Third Law, we developed a method of BIT Coding to immediately determine which questions the students responded to correctly and incorrectly on the studied N3 questions. Finally, we performed an analysis of the students' specific responses. An answer of right or wrong can supply some information about the student's conceptual understanding, but knowing exactly how the student answered ͑Which distractor did they choose?͒ allows for a more thorough assessment of students' understanding of force concepts. Therefore, the students' responses to the four N3 questions were analyzed in an investigation to determine if the students were continuing to approach problems using CS concepts or if they had transformed into Newtonian thinkers.

A. BIT coding for concept-based selected questions
The four N3 questions were established by Hestenes, Wells, and Swackhamer as questions numbered 2, 11, 13, and 14 ͓3͔. Students' responses were sorted according to the teaching method used in their introductory physics course, either JiTT or Non-JiTT and then each student's correct answer to an N3 question was quantified with a "1" and an incorrect answer with a "0." Table I gives an example of this BIT scoring for four fictional students A-D. The resulting binary number could then be transformed into its decimal equivalent. In this manner, each student's answer set was immediately quantified by a number, "15" denoting all correct answers through "0" denoting all incorrect answers and every combination in-between. Each different response combination has its own decimal value. The BIT scores were first sorted into the appropriate JiTT or Non-JiTT group and then the pretest and post-test scores were compared to identify the students that had transformed into Newtonian thinkers and those that continued to use Aristotelian concepts.

B. CS-distractor investigation
The best way to know how students are answering each question is to obtain a representative count of each specific answer to each question. All the students were again separated by teaching method, JiTT and Non-JiTT. The numbers of students giving responses a, b, c, etc. were enumerated for each question on both the pretest and post-test. Once the sums for each were obtained, separate charts were created for each question. This allowed for a simple determination of how many students answered the question correctly and the relative popularity of incorrect answers, and therefore the most prevalent pre-Newtonian beliefs.

A. Normalized gain and concept gain
The Force Concept Inventory was used in this investigation to determine if students' beliefs about Newtonian mechanics were transformed from Aristotelian beliefs to Newtonian understanding. Since the FCI was designed with this measurement in mind, an improved score on the FCI posttest shows a corresponding improvement in Newtonian thinking. To quantify this increase, a normalized gain was calculated and compared for those students taught by Justin-Time Teaching and for those students taught by more traditional methods. The JiTT group of 129 students showed a gain of 37.6% Ϯ 2.0% while the Non-JiTT group of 93 students showed a gain of 17.9% Ϯ 2.5% ͑see Table II͒. Clearly the gain of the JiTT group, which is approximately two times greater than that of the Non-JiTT group, shows that the Just- in-Time Teaching strategy enables students to better understand Newtonian concepts and compels students to eradicate many of their Aristotelian beliefs.
According to the definitions of Newtonian Entry Threshold and Newtonian Mastery Threshold developed by Hestenes, Wells, and Swackhamer upon establishment of the FCI, 39.5% of the students in the JiTT group reached the Newtonian entry threshold while only 20.4% of the students in the Non-JiTT group made a score high enough ͑Ͼ60%͒ to reach the Newtonian threshold. To reach the Newtonian Mastery Threshold, which implies that the student has dismissed all of their previous Aristotelian beliefs and thinks with a Newtonian understanding, a minimum score of 85% must be achieved. Of the students in the JiTT group, 8.5% of those students reached Newtonian Mastery, while 6.5% of the Non-JiTT group became true Newtonian thinkers ͑see Table  II and Fig. 1͒.
The Force Concept Inventory probes students' understanding of Newton's Laws, Impulse-Momentum Theory, and some kinematics. The normalized gain gives some information about how much students are learning about these Newtonian concepts. However, it is also informative to measure a concept gain, which is a gain determined from a student's pretest and post-test scores on questions pertaining to one particular concept. To measure students' understanding of Newton's Third Law a concept gain was calculated, using Eq. ͑1͒ with students' scores on the four N3 questions, for both the JiTT and Non-JiTT groups. The JiTT group showed a concept gain of 50.8% Ϯ 4.1% while the Non-JiTT group showed a concept gain of 6.6% Ϯ 5.2% ͑see Table II͒. The JiTT group's marked improvement in their understanding of Newtonian concepts when compared to the Non-JiTT group is strong evidence that the Just-in-Time Teaching method helps students to better grasp Newton's Third Law concepts when compared to students trained using more traditional techniques.

B. BIT coding results: N3 questions
Although the normalized gains and concept gains described above both provide valuable information about a student's understanding of force concepts, these measurements do not provide any information about which questions were answered correctly or incorrectly. A BIT Coding system was developed to quickly provide information about how students answered all the questions in a particular group of concept-related questions, since this information can give insight into the students' understanding of that concept. Table  III contains a key to the BIT Coding system used for the four N3 questions on the FCI, with correct answers labeled with a "1" and incorrect answers labeled with a "0." The BIT Score was determined by converting each four-digit binary number ͑nibble͒ into its respective decimal number. By applying this BIT Coding to the four N3 questions on the FCI, it was possible to denote every possible combination of answered questions with a decimal number 0-15.
These BIT scores were determined for both the JiTT and Non-JiTT groups using the students' answers to the N3 questions on the pretest and post-test. The results can be seen in Fig. 2͑a͒ for the JiTT group and Fig. 2͑b͒ for the Non-JiTT group. The pretest BIT scores ͑gray bars͒ for both groups consisted of mostly 0s and 1's, signifying that, before instruction, most students misunderstand Newton's Third Law and still rely on Aristotelian beliefs to answer questions about this concept. An interesting observation was that an unusually large number of students ͑38% of the JiTT group and 30% of the Non-JiTT group͒ had a BIT score of 1 on the pretest, which signified that these students answered question number 14 correctly but were incorrect on the other three N3 questions. However, the BIT scores of 2, 4, and 8, which also correspond to answering only one of the four N3 questions correctly, were very low on the pretests for both groups. If the students were answering one out of four questions correctly simply by chance, we would expect to see approximately equal numbers of BIT scores of 1, 2, 4, and 8. The  III. Key to the BIT Coding system, showing all the possible combinations of student's answers to N3-concept FCI questions, with "0" denoting an incorrect response and "1" denoting a correct response. BIT Coding Key No. 2 No. 11 No. 13 No. 14 Score fact that the students preferentially answered question number 14 correctly does not prove that they were using Newton's Third Law in their understanding of this question. Instead, it seems to imply a poorly written question, or a question that can be answered correctly using Aristotelian thinking. It appears as though the authors of the FCI would agree with this argument since this question has been removed from the revised version of the FCI ͑revised in August 1995 by I. Halloun, R. R. Hake, E. P. Mosca, and D. Hestenes͒. When looking at how the BIT Scores changed from pretest ͑gray bars͒ to post-test ͑black bars͒ for both groups, the percentage of students in the JiTT group with a BIT Score of 0 dropped dramatically, by a factor of 7.6, while the percentage of students in the Non-JiTT group with a BIT Score of 0 only dropped by a factor of 4.9. Interestingly, the percentage of JiTT students with a pretest BIT Score of 1 dropped by a factor of 2.4 when they took the post-test, while the percentage of Non-JiTT students with a pretest BIT Score of 1 actually increased by a factor of 1.4 on the post-test. This increased BIT Score of 1 suggests that many of the students in the Non-JiTT group could have developed a greater misunderstanding of Newton's Third Law after instruction.
Higher BIT scores generally represent that a majority of the N3 questions were answered correctly. The BIT Scores of 13 and 15 increased from pretest to post-test for both groups, which showed that students in both groups were gaining some valuable understanding of Newton's Third Law from their formal instruction. However, the JiTT group's BIT score of 13 increased by a factor of 5.7 from pretest to posttest while the Non-JiTT groups BIT score of 13 only in-creased by a factor of 2.8. Similarly, the JiTT group showed a factor of 5.9 increase in the BIT score of 15 while only 20% more students in the Non-JiTT group made a BIT score of 15 on the post-test, when compared to the pretest score. A BIT score of 15 represents correct responses on all four N3 questions, so 32% of the students taught with the JiTT method learned to think from a Newtonian perspective on the questions studied when considering Newton's Third Law problems, while only 6.5% of the students taught by more traditional means were able to transform into Newtonian thinkers.

C. CS-distractor results: N3 questions
According to the FCI published by Hestenes, Wells, and Swackhamer, question no. 2 involves a collision between a small car and a large truck and the students are asked to compare the forces that each vehicle exerts on the other. Figure 3 shows how students in both the JiTT ͓Fig. 3͑a͔͒ and Non-JiTT ͓Fig. 3͑b͔͒ groups answered this question on the pretest and the post-test. The correct response is E and the primary distractors are A, which applies the pre-Newtonian belief that a greater mass implies a greater force, and C, which applies the pre-Newtonian belief that obstacles exert no force ͓3͔. On the pretest, a majority of both groups ͑75% of both JiTT and Non-JiTT͒ showed their misunderstanding of Newton's Third Law by choosing A, which implies that, before formal instruction and given that many of the students were able to recite Newton's Third Law of "Action and Reaction," most believe that one object exerts the force ͑the "action"͒ and the other object feels the force ͑the "reaction"͒. Their common sense tells them that a bigger object will always be more forceful.
The post-test responses to question no. 2 were very different for the two groups, with 84% of the JiTT group choosing the correct response E and 68% of the Non-JiTT group continuing to choose the distractor A ͑see Fig. 3͒. The students in the classes taught by Just-in-Time Teaching are required to not only read the text before class, but they must also think critically about the text and write about the reading in their own words. It could be that this substantive and reflective writing along with the in-class discussions allow the students to move beyond memorizing that "for every action there is an equal and opposite reaction" to truly understanding the meaning of these words.
Question no. 11 involves two students sitting in office chairs and the larger student pushes with his feet on the smaller student's knees. The question compares the forces that each student exerts on the other. Figure 4͑a͒ gives the JiTT group's responses to this question on the pretest and post-test and Fig. 4͑b͒ gives the Non-JiTT group's responses. On the pretest, both groups answered this question similarly with 68% of the JiTT group and 51% of the Non-JiTT group choosing distractor D, which applies two pre-Newtonian concepts: that a greater mass implies a greater force and that the most active agent produces the greater force. Unlike question no. 2, where students were mostly divided between one distractor and the correct answer, on this question there were some students ͑23% JiTT and 15% Non-JiTT͒ had the pre-Newtonian belief that only active agents exert forces. This misunderstanding was demonstrated when they chose option B ͓3͔.
Even though the pretest responses for both the JiTT and Non-JiTT groups were fairly similar on question no. 11, the post-test responses were noticeably different for the two groups. The JiTT group showed a marked increase, by a factor of 3.6, in the percentage of students who correctly chose E on the post-test while the Non-JiTT group showed only a slight increase, by a factor of 1.3, in correct responses but also an increase in the percentage of students who chose distractor D. It is likely that this increase in the choice D is due to a continued misunderstanding of the concept of force, even after formal instruction. Before their first physics class, many students believe in the CS concept of "Active Force" ͓3͔ in which active agents have the power to cause motion. They do not believe that an inactive agent could "cause" anything. Even after instruction, many students do not understand that a force is an interaction between objects. The fact that the JiTT group showed such a great improvement on this question implies that the JiTT teaching method promotes the understanding of concepts rather than memorization. Figure 5 gives the JiTT ͓Fig. 5͑a͔͒ and Non-JiTT ͓Fig. 5͑b͔͒ responses to question no. 13 on the FCI. This question asks about the forces exerted on a large truck and a small car that happens to be pushing the truck while the car and truck are speeding up. The correct response to this question is A and the distractor ͑the most active agent produces the greatest force ͓3͔͒ is C. Again there are similarities between the pretest responses of the two groups, with 76% of the JiTT group and 71% of the Non-JiTT group choosing the distractor C. Both groups show an increase in the correct choice A on the post-test and a decrease in the number of students choosing distractor C. Although the JiTT group did have a greater percentage of students choose A than the Non-JiTT group, the majority for both groups ͑58% JiTT and 72% Non-JiTT͒ still chose the distractor on the post-test. This shows that the JiTT method is better for teaching force concepts but the students in this study are not yet Newtonian thinkers when it comes to understanding continuously applied forces.
Question no. 14 involves the same situation as that described in no. 13, however the car and truck are moving at a constant speed. Figures 6͑a͒ and 6͑b͒ give the JiTT and Non-JiTT responses, respectively, to this question, with the correct response being choice A and the distractor being choice C. As discussed in the section on BIT Coding, this question does not appear to probe the student's understanding of Newton's Third Law concepts and can be answered correctly with Aristotelian concepts. This is apparent in the pretest responses of both groups since 60% of the JiTT group and 53% of the Non-JiTT group answered this correctly without any formal instruction. After instruction, the number of correct responses increased similarly for both groups. Therefore, the results of this question do not provide evidence that one teaching method is preferred over another.

IV. CONCLUSION
By applying four different pretest and post-test analyses, it was shown that the Just-in-Time Teaching method promotes better understanding of Newtonian concepts than does traditional lecturing for introductory physics students. The normalized gain, which was calculated using pretest and post-test scores on the Force Concept Inventory for the two groups compared in this study, was more than two times FIG. 4. Student responses to FCI question no. 11 on the pretest ͑gray͒ and post-test ͑black͒. The correct response is E and the primary distractors are B and D. ͑a͒ JiTT and ͑b͒ Non-JiTT.
FIG. 5. Student responses to FCI question no. 13 on the pretest ͑gray͒ and post-test ͑black͒. The correct response is A and the primary distractor is C. ͑a͒ JiTT and ͑b͒ Non-JiTT.
FIG. 6. Student responses to FCI question no. 14 on the pretest ͑gray͒ and post-test ͑black͒. The correct response is A and the distractor is C. ͑a͒ JiTT and ͑b͒ Non-JiTT. greater for the JiTT group than for the Non-JiTT group. In addition, a concept gain was calculated for the two groups to measure the students' conceptual understanding of Newton's Third Law. The JiTT students had an N3 concept gain that was more than seven times greater than the N3 concept gain of the Non-JiTT students. These results imply that students taught by the JiTT method have a better overall understanding of Newtonian concepts than those in traditional lecture courses. This could be because, as part of a JiTT course, the JiTT students read their textbooks, think critically about the concepts and then reflectively write on the concepts all before the concepts are even discussed in class. This critical thinking and reflective writing helps the students actively participate in class discussions during which their previous Aristotelian beliefs are eradicated and Newtonian conceptual understanding is established. Support for the JiTT teaching method over traditional lecture was reinforced with BIT Coding and CS-Distractor analyses, which also showed a greater improvement in N3 conceptual understanding for the JiTT group over the Non-JiTT group. Furthermore, the BIT Coding analysis was useful in determining that one of the FCI questions, question no. 14, is not valid in testing a student's understanding of Newton's Third Law, since most of the students in both groups were able to answer this question correctly on the pretest while they incorrectly answered the other N3 questions. In future studies, the BIT coding analysis will be used for other concepts tested with the FCI to check the validity of other conceptual questions within the instrument, as well as to measure student's learning of Newtonian concepts after they undergo a university-level introductory physics course.