Modeling instruction: Positive attitudinal shifts in introductory physics measured with CLASS

Among the most surprising findings in Physics Education Research is the lack of positive results on attitudinal measures, such as Colorado Learning Attitudes about Science Survey CLASS and Maryland Physics Expectations Survey MPEX . The uniformity with which physics teaching manages to negatively shift attitudes toward physics learning is striking. Strategies which have been shown to improve conceptual learning, such as interactive engagement and studio-format classes, provide more authentic science experiences for students; yet do not seem to be sufficient to produce positive attitudinal results. Florida International University’s Physics Education Research Group has implemented Modeling Instruction in University Physics classes as part of an overall effort toward building a research and learning community. Modeling Instruction is explicitly designed to engage students in scientific practices that include model building, validation, and revision. Results from a preinstruction/postinstruction CLASS measurement show attitudinal improvements through both semesters of an introductory physics sequence, as well as over the entire two-course sequence. In this Brief Report, we report positive shifts from the CLASS in one section of a modeling-based introductory physics sequence, for both mechanics N=22 and electricity and magnetism N=23 . Using the CLASS results and follow up interviews, we examine how these results reflect on modeling instruction and the unique student community and population at FIU.


I. INTRODUCTION
Significant attention has recently been paid to assessing students' attitudes, expectations, views, and epistemological beliefs because they are believed to play important roles in learning and distinguish experts in a field from novices. [1][2][3][4] A striking outcome from the development of such attitudinal surveys is the consistent negative shifts shown by students throughout introductory physics courses. 1,2 The overall negative shifts seem independent of instructional approach, even when other measures such as normalized gain on the Force and Motion Concept Evaluation ͑FMCE͒ indicate the course has successfully addressed conceptual learning. 1 Conceptual understanding is one of many characteristics that distinguish experts from novices, but other attitudinal characteristics discriminate novice and experts as well. These attitudes are not merely a like or a dislike of physics, but a broader evaluation of cognitive attitudes toward the nature of physics and the practice of physics. Expert attitudinal characteristics include viewing physics as a coherent, connected group of topics and seeing problem solving as a conceptually grounded search through the knowledge base, rather than as a hunt for equations. Reformed teaching seeks to enhance the development of expertlike characteristics in students. 5,6 Reform based strategies, such as active engagement and studioformat classes, have been shown to improve conceptual learning as compared to traditional instruction. 7,8 A reasonable conjecture is that simply improving conceptual learning is sufficient to develop expertlike characteristics in students; however, the overall negative shifts on attitudinal surveys indicate the contrary.
At the heart of our science education endeavor is the desire to nurture scientific curiosity and capacity in our students, preparing them as both future scientists and citizens.
National science standards reflect the value placed on classes that provide students with authentic science experiences, in order that students gain an appreciation for the nature of science. 9 Common instructional approaches to addressing the nature of science include using inquiry-based methods and encouraging hands-on, minds-on teaching. Again, approaches which aim to engage students in authentic science experiences and therefore the nature of science should be believed to improve attitudinal measures toward science and science learning, and again, the lack of positive results in introductory physics is striking. Otero and Gray have the first published positive results using CLASS in courses for preservice teachers that emphasize the nature of science and utilize the Physics and Everyday Thinking ͑PET͒ and Physical Science and Everyday Thinking ͑PSET͒ curricula. 10 Redish and Hammer found positive shifts using the Maryland Physics Expectations Survey ͑MPEX͒ II in an introductory physics course for biology students using curriculum materials which emphasize the nature of science as an epistemological framing. 11 Florida International University's ͑FIU͒ Physics Education Research Group has implemented modeling instruction in several sections of introductory calculus-based physics as the central educational reform effort associated with the Center for High Energy Physics Research and Education Outreach ͑CHEPREO͒. CHEPREO is one of several coordinated efforts at FIU with the intention of improving participation by traditionally underrepresented minority and women students, primarily in physics, but also in other sciences. 12 The ongoing assessment of the Modeling Instruction sections includes administration of the CLASS as a pre-or postdiagnostic each semester. Among the results we present in this Brief Report are the positive overall shifts as measured by the CLASS, the positive shifts in the first semester on four of eight CLASS categories, and the progression of CLASS scores for students enrolled in sequential semesters of one section of introductory physics. In order to corroborate the positive shifts, we have conducted interviews with students in the Modeling Instruction courses. The findings of these interviews will also be presented.

II. MODELING INSTRUCTION AT FIU
FIU is the largest source of Bachelors degrees for Hispanic students in the United States. It is a large, public urban research university with 38 290 students with a Hispanic enrollment of nearly 60%, reflective of South Florida's demographics. The student population at FIU makes it an ideal setting for increasing participation of Hispanic students in physics while developing models for improving the educational landscape for all students.
Modeling instruction is a reform effort that has had great success at the high school level, 13 and which is based on the Modeling Theory of Science. This epistemological basis is evident in the curriculum as student activities are focused on the process of building, validating, and deploying models. This process of modeling replicates the central activity of practicing scientists and, therefore, strongly integrates an explicit nature of science theme throughout the curriculum. 14 The implementation of Modeling Instruction at FIU is designed to give students an authentic scientific experience and to make the nature of science a coherent theme across content and pedagogy. The Modeling Instruction course operates as a collaborative learning environment, with 30 students in a studio-format class with integrated laboratory and lecture. Inquiry laboratories and activities focused on conceptual reasoning and problem solving are the primary vehicles through which models are built, validated, and extended. 15 The instruction in the particular section we are investigating utilizes Modeling Discourse Management, 16 a technique for directing student-student discourse. Students, in small groups, work on activities designed to encourage model building. Students share their ideas via portable whiteboards, coming together for student-driven discussions. The instructor's role is to moderate discussion and orchestrate appropriate activities for conceptual development. The conceptual learning gains are consistent with other reform pedagogies, with a FCI normalized gain of 0.43 ͑with an average prescore of 34.0% + / −3.3%.͒ Although there are three sections of Modeling Physics each semester, Modeling Discourse Management is used only in the section we are investigating.

III. METHODS
We have used the CLASS, an instrument developed and validated at University of Colorado, to measure students' attitudes toward science and learning science. The CLASS consists of 42 statements to which students respond that they agree or disagree using a five-point Likert scale. 36 of the 42 statements are scored by comparing the student's response to the expert response. One statement, No. 31, is used to eliminate surveys from students not carefully reading the statements. Adams et al. 1 identified overall expert responses as well as eight subcategories ͑identified in Fig. 1.͒ An individual overall favorable score is calculated for each student as the percentage of the student's answers that match the expert response. The overall favorable score for the whole class is determined by averaging these individual student scores. For more details see Ref. 1.
The particular Modeling Instruction course examined in this study ran during the 2007/08 academic year and followed the standard sequence with mechanics during fall and electricity and magnetism during spring. The fall enrollment was 30 students. The spring enrollment was 31 students, 26 of whom continued from the fall. The class met 3 days a week, and each meeting lasted 2:15. The CLASS was administered on the first day of each semester and again during the last week of the semester. Two conditions were necessary for inclusion in the analysis: all surveys were matched pre-or postpairs and students who did not answer statement No.31 correctly were excluded. During each semester, we have a high percentage of matched pre-and post-tests ͑22 of 30 in mechanics, 23 of 31 in electricity and magnetism͒. Further, 26 students completed the entire two semester sequence, of these 16 have matched pre/post for each semester providing us an opportunity to evaluate the progression of CLASS results over an entire year. We report the data from each semester and from the full year separately.
Data were analyzed using the standard analysis template provided by the developers of the CLASS. The template provides overall and categorical results as well as shifts from pre to post. Shifts of pre to post scores were calculated for each combination of pre-and postdata by taking the difference. We focus on three combinations: pre/post within each semester and over the entire year ͑i.e. fall pre to spring post͒. The significance of the shifts was determined by comparing the average of the individual shifts, ͑post-pre͒ avg , to the standard error of these shifts. 17 In addition to administering the CLASS on paper, follow-up interviews were conducted with students in the Modeling Instruction class. Interview volunteers were requested from the entire class, 12 students volunteered, and 7 students participated in interviews. We conducted three interviews in groups of two to three students at the end of the spring term. Each interview was conducted according to a standard protocol which included background questions about their experiences in their physics class, followed by discussion of statements from the CLASS survey. During the interview, students were presented with a copy of the CLASS with their responses, the students first read the question aloud, reported the answer they selected and were asked to explain the reasoning behind their answers. The reasoning students provided generally related to their experiences in the Modeling class. Interviews were video recorded and transcribed, then characteristic answers were compiled from the transcripts.

IV. RESULTS AND ANALYSIS
The Modeling Instruction section had significant positive shifts toward more favorable responses overall, as well as in four of the eight subcategories, during the fall semester ͑N =22͒; see Fig. 1. Shifts in two categories were significant at the p Ͻ 0.05 level ͑Problem Solving General and Conceptual Connections͒; the PS Sophistication category was significant at p Ͻ 0.01, and Applied Conceptual Understanding was significant at p Ͻ 0.001 level. The Overall shift is significant at the p Ͻ 0.01 level. During the spring semester ͑N =23͒ only the Personal Interest shift is significant ͑p Ͻ 0.01͒; see Fig. 2. Figure 3 shows the progression of scores for the 16 students who completed the CLASS pre and post during both semesters. Examining the shifts over the entire course, all the shifts are significant except for the Real World Connections. Additionally, the Overall shift in favorable responses is large ͑12%͒ and positive ͑p Ͻ 0.001͒.
The data summarized in Figs. 1 and 2 indicate that during each of the two semesters, students' attitudes about learning science shifted toward more expertlike. Further, as can be seen in Fig. 3, the scores rise over the duration of the instruction, with little change during the winter break.
The basic interpretation of the CLASS data indicate that for one stable, although small, population of students in Modeling Instruction physics there is a shift toward more expertlike attitudes in science over two semesters. The shifts appear linked to the instructional approach, although the class profile starts higher than published data, still it increases during both semesters. These results from this Modeling Instruction course are consistent with the findings of Otero, 10 where an inquiry-based approach, similar to Modeling Instruction that includes an explicit focus on the nature of science, was found to improve student attitudes toward science and the learning of science.
The student interview responses to the background questions and CLASS statements are consistent with the interpre- tation that the emphasis on the nature of science, explicit and inherent in Modeling Instruction, influenced students and likely led to the improvements on the CLASS. James, a second year engineering student, described the benefits of the Modeling Instruction class as follows, "The experiments in class and being able to figure everything out on our own kind of felt like actual research, and then doing the whiteboards is like publishing a paper, I guess. So working in like a scientific environment." Later in the interview, James described the utility of making whiteboards and validating them with peers by acknowledging that he carries a small whiteboard around to work on homework, "…it gives us an idea of what's really happening." Katie, a first year premed major identified the role of whiteboarding and the process of building models as the most important element of the class. "Making the models and pretty much teaching yourself is what really helped me…You have to make a model, go through, come up with your own conjectures and your own equations." The role of models was prevalent, especially as students talked about solving physics problems. Again Katie describes the utility of doing lots of problems, "I guess it is helpful to do lots and lots of problems, but also I found it helpful to get the most difficult problem out of every homework and do it in detail." Katie identifies that building a detailed model of one situation is as useful or more useful in learning. Lenny, a second year student described how the whiteboards helped them construct models, "We would whiteboard the concepts because we would be given an experiment…and we'd get results and have to put them down on the board and discuss them with the entire class…to see if we were right and that really helps you. In the Modeling class we get a chance to discover it for ourselves and the whiteboarding was the initial way we would figure out these models." In addition to supporting the creation of models, whiteboards transcend the classroom. Because the in-class interactions are collaborative and focused on building models, students continued this practice outside of the class. Lorraine, a second year student describes how the in-class interactions encourage further out-of-class interactions, "One of the biggest benefits ͑of the Modeling class͒ was that outside of the classroom was that you get to know people in the classroom because of the way the class is conducted so that encourages you to study with them outside." A limited number of interviews with students after two semesters of Modeling Instruction indicates that the instructional approach impacted the way they approached learning, the way they viewed science, and the nature of their interactions when learning with peers. The interview participants became more active agents in the learning process, relying less on the instructor and more on their peers by utilizing whiteboards to create models that are robust and adaptable to different situations. The in-class collaborative learning environment, centered on building models encouraged them to collaborate in similar ways outside of class. Student responses indicate that Modeling Instruction impacted cognitive attitudes; consistent with our hypothesis that the pedagogy is the primary source of attitudinal shifts.

V. DISCUSSION AND CONCLUSIONS
We have reported the first significant positive attitudinal shift for an introductory physics course. The results come from a single class over two semesters and it would be premature to generalize our results at this time, especially as other courses that can be characterized as interactive engagement have failed to yield positive shifts. Further, our data cannot separate the effect of the instructor from the effect of the pedagogy. However, the results cannot be dismissed and the positive CLASS shifts and the nature of these shifts indicate that Modeling Instruction is impacting students in a significant, positive manner. These results are substantiated by interviews with students in the Modeling Instruction class. These data are viewed as a motivation for further study. A comprehensive follow-up research project, involv- ing additional instructors in Modeling physics as well as traditional students in introductory physics, is being planned. The research will include both attitudinal surveys as well as interviews.
Two interesting aspects of the data deserve further discussion. First, the preinstruction averages are higher than published results. 1 One explanation for the higher preinstruction average is the existence of a selection effect. Modeling classes are wildly popular, with requests outpacing availability by a factor of four. Clearly, the course has a favorable reputation for being different than the standard lecture course, and this reputation may lead to more favorable attitudes prior to instruction based on students choosing the Modeling Instruction class. The selection effect, however, is not reflected in conceptual understanding measures; students in the modeling courses start with FCI prescores slightly lower or equivalent to students in comparable courses. Even with the initial favorable attitudes, the class achieves a significant improvement in attitudes. Future research efforts will target identifying the source and role of the preinstruction favorable attitudes by examining introductory physics students in other Modeling Instruction sections as well as in traditional classes.
Second, the data afford an investigation of the progression of attitudinal shifts over two sequential semesters with a population that remains mostly intact for the second semes-ter. Students in the Modeling Instruction course show significant positive shifts after a single semester overall and in four of eight categories. This indicates that a single semester is sufficient to significantly shift attitudes of students. We find in this case that students do not shift significantly during winter break and conclude that attitudes measured by the CLASS are stable over extended periods of time.
The compelling nature of these results merits further study. Additional data, including data from other instructors, will allow us to further test our hypothesis and investigate causal links between course structure and impact on student beliefs paying special attention to the role of Modeling Discourse Management in the Modeling classes. As we extend the scope of this investigation, we will be collecting data from comparable student groups to further substantiate the claims made in this preliminary study.