Attitudinal gains across multiple universities using the Physics and Everyday Thinking curriculum

Instructional techniques based on research in cognitive science and physics education have been used in physics courses to enhance student learning. While dramatic increases in conceptual understanding have been observed, students enrolled in these courses tend to move away from scientistlike views of the discipline and toward novicelike views, as measured by various assessment instruments. It has been proposed that course materials and instruction that explicitly address epistemology, the nature of science, and the nature of learning science will help students develop views more closely aligned with the views of scientists. The Physics and Everyday Thinking PET curriculum has specific goals for helping nonscience majors explicitly reflect on the nature of science and the nature of learning science. We show that in PET courses with small and large enrollments, shifts toward expert responses ranged from +4% to +16.5% on the Colorado Learning Attitudes about Science Survey. These results are compared to results from other studies using a variety of similar assessment instruments.


I. INTRODUCTION
The work in this paper addresses the national need for increased scientific literacy among the general public as well as increased science literacy among future and practicing elementary and middle school teachers. 1,2Among the issues raised in national reports that focus on these issues is the need for undergraduate science courses that not only address fundamental content goals but also explicitly address the nature of scientific knowledge, science as a human endeavor, and the unifying concepts and processes of science.Researchers and curriculum developers have responded to this call by developing inquiry-based physical science curricula especially for the postsecondary nonscience major population.Such curricula include Physics by Inquiry, 3 Powerful Ideas in Physical Science, 4 Workshop Physical Science, 5 Operation Primary Physical Science, 6 Physics and Everyday Thinking, 7 and Physical Science and Everyday Thinking. 8In most cases, large conceptual learning gains have been found to be associated with these specialized curricula. 6,9,10However, little work has been done to determine how such courses might impact students' views about science, their views about learning science, and their understandings of various aspects of the nature of science.
Research has consistently demonstrated that K-12 teachers and their students do not develop desired understandings of the nature of science. 11"The nature of science" is a phrase used to distinguish scientific knowledge from other ways of knowing through "the use of empirical standards, logical arguments, and skepticism." 1 Abd-El-Khalick 11 showed that while preservice elementary teachers have difficulty learning about the nature of science, they develop a better understanding of certain aspects of the nature of science when nature of science instruction is embedded within instruction on specific scientific content.Yet, few science teacher education courses focus on content and few science content courses focus on the nature of science and the nature of learning science.Not only does instruction on the nature of science and the nature of learning science need to be embedded in content, it also needs to be explicit in order to be effective. 12nstruction that explicitly addresses these issues has been shown to be much more effective in improving students' views and attitudes about aspects of the nature of science and science learning than implicit approaches that use inquirybased science activities but lack explicit references to the nature of science or issues in the nature of learning science. 12or example, Elby 13 demonstrated that students enrolled in a curriculum that was "epistemologically focused" showed significantly greater gains on the Epistemological Beliefs Assessment for Physical Science 14 than did curricula focused on content only.Elby argued that "even the best curricula aimed at conceptual development but not aimed explicitly at epistemological development do not produce comparable epistemological results ͑p.S57 of Ref. 13, emphasis Elby's͒." Several instruments have been developed to assess students' views of the nature of science and the nature of learning science.These include the Maryland Expectations ͑MPEX͒ Survey, 15 the Epistemological Beliefs Assessment for Physical Science ͑EBAPS͒, 14 the Colorado Learning Attitudes about Science Survey ͑CLASS͒, 16 and the Views of the Nature of Science ͑VNOS͒ Survey. 11While each of these instruments measures slightly different aspects of students' expectations, beliefs, and understandings of the nature of science and of learning science, studies that use these instruments show remarkable similarities-students of all ages have difficulty learning how science knowledge is constructed and in most cases regress in sophistication as measured by these instruments over a semester-long science course.For example, Redish et al. 15 found that at six different schools, students' overall MPEX scores deteriorated over a semester of introductory physics rather than becoming more sophisticated.Although students enrolled in courses that use research-based curricula show significant conceptual gains as measured by conceptual instruments such as the Force and Motion Conceptual Evaluation ͑FMCE͒, 17 they do not show significant increases in beliefs about aspects of the nature of learning science as measured by the MPEX or the CLASS and, in fact, show negative shifts on the CLASS. 18hese results suggest that conceptual learning is not necessarily associated with the development of expertlike thinking about the nature of learning science.Hrepic et al. 6 studied a curriculum designed specifically for preservice elementary teachers, the Operation Primary Physical Science ͑OPPS͒ curriculum, from both the conceptual and attitudinal perspectives.While significant conceptual gains were found using a conceptual instrument designed by the authors of the curriculum, only very small positive shifts in attitudes about science and the nature of learning science were measured by the CLASS; and in one of three courses studied, CLASS scores deteriorated 6 over a single semester.Although the OPPS curriculum was developed for preservice elementary teachers, it does not appear to explicitly address issues about learning science and the nature of science knowledge.Rebello and Mamolo 19 also studied a research-based curriculum designed specifically for preservice elementary teachers.Students in this learning cycle-based course were given both the EBAPS ͑Ref.14͒ and VNOS ͑Ref.11͒ surveys.Neither survey showed significant shifts in students' overall scores on these instruments, although the EBAPS showed a small positive shift in the real-life applicability category and the VNOS showed a small positive shift in the inferential nature of models.
We hypothesized that the Physics and Everyday Thinking curriculum and the Physical Science and Everyday Thinking curriculum would show gains on an instrument that purports to measure attitudes about learning and students' views of the nature of science because both curricula explicitly address these issues.The remainder of this paper will report on a study in which this hypothesis was investigated.

Physics and Everyday Thinking ͑formerly Physics for Elementary Teachers͒ and Physical Science and Everyday
Thinking are inquiry-based curricula designed to meet the needs of elementary teachers.Physics and Everyday Thinking ͑PET͒ course content focuses on the themes of interactions, energy, forces, and fields.Physical Science and Everyday Thinking ͑PSET͒ is based on the PET curriculum but integrates significant chemistry content by focusing on PET themes plus atomic-molecular theory to account for physical and chemical changes, conservation of matter, and gas behaviors.Each curriculum consists of carefully sequenced sets of activities intended to help students develop physical science ideas through guided experimentation and questioning with extensive small-group and whole-class discussion.The curricula also include a series of learning about learning ͑LAL͒ activities, in which students are explicitly asked to reflect on their own learning, to reflect on the learning of other students, and to reflect on the learning of scientists.The LAL activities are embedded throughout the curricula, often occurring between two content-focused activities.For example, after PET students develop a domainlike model of magnetism through experimentation and consensus discussions, they read about the historical development of the domain model of magnetism and investigate how and why the scientifically accepted model of magnetism changed over time.PET and PSET students study their own learning by reflecting on how their ideas about a particular concept changed over time and what classroom activities, discourse, or experiments influenced these changes.Finally, PET and PSET students investigate the learning of others by watching short video clips of elementary children struggling with scientific issues similar ͑but age appropriate͒ to those found in the PET and PSET curricula.The PET and PSET learning about learning activities are similar in format to the content activities and follow a guided-inquiry format.Each activity has three parts: ͑1͒ initial ideas, ͑2͒ collecting and interpreting evidence, and ͑3͒ summarizing questions.Examples of the table of contents for Chap.6 in PET and Chap. 4 in PSET are shown in Fig. 1.In Fig. 1, LAL activities are indicated with a ࡗ symbol.In PET Chap.6, activities 1 and 2 focus on children's ideas, activity 2 homework focuses on PET students' own ideas, and activity 4 focuses on social scientist ideas about learning.In PSET Chap.4, activity 3 homework focuses on children's ideas, activity 4 focuses on students' own ideas and how science ideas are represented by the computer simulator, and activity 5 homework focuses on the nature of science.
PET and PSET explicitly address issues about the nature of science and the nature of learning science through the LAL activities, and these activities are embedded within the content instruction.Thus, it would be expected that students who participate in the PET and PSET curricula would be more likely to move toward expertlike thinking about the nature of science and the nature of learning science than  Previous work using the EBAPS at one institution showed that PET and PSET students did indeed have higher positive shifts toward expertlike responses than did students from traditional courses. 20The EBAPS was administered pre and post to 228 students enrolled in traditional lecture-format earth science and general physics courses and to 173 students enrolled in PET and PSET courses at a single university.Posttest composite scores were 14% to 25% higher for the students enrolled in the PET and PSET courses than for the students enrolled in the traditional courses.Furthermore, composite scores indicate a ϩ7% to ϩ18% change toward "expertlike" responses about the nature of science and learning for students enrolled in the PET and PSET courses and a change of −2% to −9% for students enrolled in the traditional courses.In other words, the traditional students became more novicelike in their responses about the nature of science and learning during a semester of traditional instruction, but the PET and PSET students became more expertlike in their responses.The trend toward novicelike responses over a single semester is commonly observed in science courses, even those that are based on research in science education.However, Elby 13 found overall gains as high as 5.3% on the EBAPS when he used curricula that were epistemologically focused.
In Sec.III we discuss the administration and analysis of the CLASS survey in PET and PSET courses.The atypical shifts found among students enrolled in PET and PSET may be the result of the intensive explicit focus on issues of learning about learning embedded throughout the curricula.

III. METHOD AND DATA ANALYSIS
We administered pre-and post-CLASS surveys to 360 students enrolled in nine different PET and PSET courses at seven universities.These PET and PSET courses had enroll-ments ranging from 13 to 100 students.The CLASS was chosen for this study because it is worded to be applicable to a wide variety of physics courses and to be meaningful to students who have not taken physics. 16A total of 288 presurveys and 265 postsurveys were returned.The conditions for the administration of the surveys were similar across universities.All were done online.Most were for extra credit and were done outside of class.Of the surveys returned, 182 were used for the analysis because they met the following criteria: the student submitted both pre-and postsurveys, the student correctly responded to question 31 ͑which is intended to catch students who are not reading the survey͒, the student did not select the same answer to almost all the statements ͑which is an indication that the student did not read the survey͒, and the student answered all or almost all of the statements.Characteristics of each of these universities and courses are shown in Table I.One university offered three courses taught by two different instructors.Institutions ranged from community colleges to research universities and were located throughout the United States.
The CLASS consists of 42 statements for which students are asked to agree or disagree using a five-point Likert scale.Of these 42 statements, 36 are scored by comparing a student's response to the expert response.Statement 31 is not scored but is used to eliminate some surveys as described above.The other five unscored statements lack an expert consensus and are being revised.An overall favorable score is then calculated for each student by comparing the percentage of answers in which his or her response matches the expert response.An average score for each course is then found by averaging all student scores.We used the standard CLASS scoring procedure reported by Adams et al. 16 Most questions on the CLASS are categorized into at least one of the eight categories listed in Table II with sample questions.III along with the shifts for each course, which range from +4% to +16.5.Aggregate results from the CLASS study show an average shift of 8.8% in PET and PSET courses compared to average shifts of −6.1% to +1.8% found in other physical science courses ͑OPPS͒ designed es- pecially for elementary teachers with enrollments ranging from 14 to 22 students. 6Interestingly, the PET and PSET courses with the largest pretest scores had the smallest overall positive shifts.Courses with a statistically significant shift in average score from pretest to posttest are indicated in Table III with an asterisk.A t test was used to test for significance and to test the normality of each data set and an r-squared value was obtained for the data from each course by fitting a line to a plot of the theoretical Gaussian curve and the observed data points.The r-squared values ranged from 0.82 to 0.99, so it is reasonable to report the standard error and p value for each course, although the sample size was small in some cases.The data from Table III are rerepresented in Fig. 2 with data from a study on second-semester calculus-based introductory physics courses taught in the same format at a single university. 18The negative CLASS shifts shown for the calculus-based courses in Fig. 2 are representative of physics courses and are small in comparison to traditional courses.Each of the calculus-based courses except Fall 2005 and Fall 2006 was taught by at least one faculty member who was experienced in methods supported by physics education research.Figure 2 highlights the difference in CLASS shifts among PET and PSET students in comparison to a transformed calculus-based course taught by different instructors. 18able IV shows comparisons of CLASS pre and post scores and shifts for different curricula and different instructional settings.As shown in Table IV, the average CLASS pretest score is 54% for the PET and PSET courses, which is lower than typical scores found in calculus-based courses.Traditional lecture style courses have been found to have shifts on the CLASS of −8.2% to +1.5% in calculus-based courses with enrollments of 40-300 students in each course section.Shifts of 1.0% have been found in courses for nonscience majors and −9.8% in algebra-based courses for premedical students. 21Of the courses shown in Table IV, only PET and PSET embed activities that explicitly address the nature of science and the nature of learning science.

Pre-and post-CLASS scores for each participating course are shown in Table
Recent research has demonstrated significant positive shifts ͑4.6% to 11.7%͒ on CLASS scores in some Physics by Inquiry courses that have a strong implicit focus on the nature of science and the nature of learning science but do not explicitly address these issues. 22The same study did not show positive shifts in other Physics by Inquiry courses investigated.
Figure 3 shows the PET and PSET CLASS scores according to the categories shown in Table II.The shifts in all categories except sense making or effort and problemsolving ͑PS͒ confidence were significant at p Ͻ 0.01.PS confidence was significant at p Ͻ 0.05.The highest shifts for PET and PSET students were made in the following categories: personal interest, problem-solving sophistication, conceptual understanding, and applied conceptual understanding.These categories are similar to one another but have subtle differences.Overall, these categories assess students' beliefs about the structure of knowledge ͑i.e., is it a set of isolated facts handed down from authority, or connected concepts that should make sense to the learner?͒.These categories also assess students' views on the extent to which one's conceptual understanding ͑in contrast to formulaic manipulation and memorized facts͒ can and should be used in the process of solving physics problems.
In the postsurvey, students were asked additional questions about their interest in physics, using the following questions: ͑i͒ At the start of the term what was your level of interest in physics?͑Low, moderate, high͒ ͑ii͒ Currently what is your level of interest in physics?͑Low, moderate, high͒ ͑iii͒ During the semester, my interest in physics ͑increased, decreased, stayed the same͒.
In response to these questions 59% of PET and PSET students rated their interest in physics as low when they began the course, while only 21% rated their current interest as low at the end of the semester, as shown in Fig. 4͑a͒.Of these students, 66% said that their interest in physics increased during the semester.In a study involving a more traditional calculus-based physics course, Perkins et al. 21ound an increase of only 19% and a decrease of 45% on the same questions ͓shown in Fig. 4͑b͒ for comparison͔.As is shown in the two graphs in Fig. 4, the PET and PSET curricula seem to have the opposite effect of the calculus-based course shown.
The questions associated with the categories of personal interest, problem-solving sophistication, conceptual understanding, and applied conceptual understanding seem to be related and interdependent as viewed by the students.For example, in a previous semester, PET students were asked why they stated that their interest in physics increased, de- creased, or stayed the same.Some representative statements are listed below.
"This class showed me how to solve physics problems on my own taking into consideration my own experiences.I learned that physics is neither scary nor is it "all" complicated.Many of the problems and the demonstrations of how things work turned out to be very simple.I loved the way the class was taught-I learned everything through discussions and exploration and discovery." "This class showed me how cool physics can be… cool in the sense that physics is such a huge part of my life… and cool in the sense that I feel like I have a solid understanding of the topics we explored." "It was like the 'blinders' were removed.I feel like I look at the world around me totally different than I did before.I now look at situations or scenarios and try to explain or make sense of them scientifically rather than accepting them at face value." As is suggested by the representative statements above, interest in physics appears to be associated with the students redefining what physics problems are, how problem-solving relates to the self, and that physics is not scary and foreign but a part of one's everyday life.
These students were also asked why their confidence in taking a future physical science course increased, decreased, or stayed the same.The following answers are representative.
"I can see now that physics is more than just solving equations.The concepts are much more important.I can use this to help think about difficult problems.""I feel like I could at least try to figure out any physics problem that was put before me.I may not solve it but I would at least know how to tackle it." "I understand the concepts behind the mathematical equations.I can now build on this understanding.Prior to this class, I memorized bits of information and equations or vocabulary but did not truly understand the concepts represented in them." These statements suggest that students' confidence in solving physics problems might be tied to their recognition of problem solving as involving concepts that these students came to believe could be understood.These students seem to be explicitly aware that solving physics problems conceptually is more than just the stating of memorized facts and vocabulary.

V. CONCLUSIONS AND IMPLICATIONS
The answers PET and PSET students gave in the CLASS presurveys suggested that they were thinking about solving physics problems as the process of arriving at a predetermined answer through formulaic manipulation and memorization.Their answers on the postsurveys suggest that the students were now thinking about solving physics problems as the process of making sense of physical phenomena.In all PET and PSET activities, students are expected to first express their initial ideas about a physical phenomenon, then engage in an experiment or manipulate apparatus to collect and interpret evidence, and finally they answer summarizing questions designed to help them make inferences from their data.This learning sequence is different from what they might have experienced in other high school or college-level physics courses that rely on lectures and note taking.It seems that the experience of working with apparatus and generating explanations in small and large groups has allowed the students to obtain a more personal connection to the content of the course.We conjecture that, in a sense, students in PET and PSET classes redefined for themselves what it means to solve problems in physics.
The conceptual nature of the course alone could have led to the atypical positive shifts on the CLASS found in this study.However, Akerson et al. 12 and Elby 13 argued that explicit instruction on the nature of science and on the nature of learning is necessary if we expect to see students make these types of gains.Abd-El-Khalick 11 further argued that this type of instruction must be embedded in content-focused instruction.While the PET and PSET curricula have all of these elements, further work is needed to distinguish the effects of the conceptual focus of the curriculum, the explicit instruction on the nature of science and the nature of learning science, and the fact that the explicit instruction is embedded within the content.Regardless of the causal variable or combination of variables, this study has demonstrated that the PET and PSET curricula are effective in creating positive shifts among nonscience majors in their responses to the CLASS survey.
A large portion of the students who enroll in a PET or PSET course are prospective and practicing elementary or middle school teachers.Future and practicing teachers must develop positive views of science since their views will greatly influence the views of their students.Elementary teachers are often the first representatives of the science community with whom young children come into contact.These teachers can encourage or discourage their students from scientific inquiry or they can avoid the topic of physical science altogether.The PET and PSET curricula seem to help prospective and practicing teachers enrolled in the courses develop confidence in their ability to do and teach physical science.Students enrolled in these courses appear to learn to see themselves as involved in personal sense making when they are engaged in the problem-solving process.In doing so, they develop more sophisticated views of the nature of science and the nature of learning science.This is a central goal of scientific literacy for the general public.
Table of contents of Chap.6 PET and Chap. 4 PSET.studentswho participate in physics curricula that either address the nature of science and science learning implicitly or do not attend to them at all.To investigate the hypothesis that students in PET and PSET courses would move toward expertlike thinking about the nature of science throughout one semester of taking PET or PSET, we administered the CLASS instrument in multiple PET and PSET classroom settings.

TABLE I .
Characteristics of schools and courses.
aStudents in these courses were divided into two different sections.

TABLE II .
Categories of the CLASS instrument with example questions.

TABLE III .
Overall favorable scores on CLASS ͑standard error in parentheses͒.
a p value Ͻ0.05.

TABLE IV .
Comparison of overall CLASS scores for different courses. .2. ͑Color͒ CLASS shifts for PET and PSET courses compared to calculus-based courses; PET and PSET courses are numbered to match corresponding data in Table III.FIG. 3. ͑Color͒ PET and PSET CLASS scores by category.FIG. 4. ͑Color͒ Personal interest in physics in ͑a͒ PET and PSET and ͑b͒ calculus-based course.
a Pre and post scores were not reported for these courses.FIG