Energy Project professional development : Promoting positive attitudes about science among K-12 teachers

Promoting positive attitudes about science among teachers has important implications for teachers’ classroom practice and for their relationship to science as a discipline. In this paper, we report positive shifts in teachers’ attitudes about science, as measured by the Colorado Learning Attitudes about Science (CLASS) survey, over the course of their participation in a professional development course that emphasized the flexible use of energy representations to understand real world scenarios. Our work contributes to the larger effort to make the case that professional development matters for teacher learning and attitudes.

The Energy Project at Seattle Pacific University was a six-year, NSF-funded project with the primary aim of promoting K-12 teachers' development of formative assessment practices in the context of energy.In designing professional development (PD) courses to accomplish this goal, our project team paid attention to additional design criteria, including (1) what the structure and content of our course communicated about what science is and (2) how we might create an environment in which teachers felt capable of learning (and doing) science.Our consideration of these additional criteria led us to wonder if our PD courses would measurably promote positive attitudes about science among teachers.This question was tied in part to research (e.g., Refs.[13,25]) that suggests that courses with a focus on epistemology (i.e., knowledge generation practices in science) can contribute to shifts in learner attitudes.It was also tied to the presence of questions about one's sense of self as capable of learning science on attitudinal surveys (e.g., the "effort" category on the Colorado Learning Attitudes about Science Survey [12]).This led us to ask the following research question, in the context of a PD course offered in the summer of 2013: Do teachers express more expertlike attitudes about science after Energy Project professional development than before?
Researchers have defined "attitudes" in many different ways-a variety that is reflected in the citations above.In this paper, we align our sense of attitudes with that represented in the Colorado Learning Attitudes about Science Survey (CLASS) [12], which measures "beliefs about physics and about learning physics."We elaborate on this in Sec.III.
We situate our work within the literature on teacher professional development.Much of this literature explores PD design or features of effective PD (e.g., Refs.[26][27][28][29][30][31][32][33][34][35]); this paper takes an empirical approach, showing that our PD was effective along dimensions that mattered to us.Our work aligns with what Borko [33] calls "phase 1 research," which studies an individual professional development program at a single site.Borko points out that the "goal of phase 1 research" is often to "create an existence proof; that is, to provide evidence that a professional development program can have a positive impact on teacher learning."Together, phase 1 research projects cohere to make the case that PD matters for teacher learning and development.The research in this paper contributes to making the case that PD matters for teacher attitudes; to our knowledge, this is the first paper that uses attitudinal surveys from physics to measure in-service teacher attitudes about science in the context of PD.
We introduce our professional development context in Sec.II and describe our research methods in Sec.III.We present our results in Sec.IV.We close by discussing our results and sharing implications of our work in Sec.V.

II. ENERGY PROJECT PROFESSIONAL DEVELOPMENT A. Professional development approach
Energy representations.-EnergyProject professional development courses make extensive use of a suite of energy representations called Energy Tracking Representations (ETRs), including an embodied learning activity (Energy Theater [36]) and diagrammatic representations (Energy Tracking Diagrams [37] and Energy Cubes).These representations are designed to support teachers in using and refining a theoretical model for energy that emphasizes conservation, localization, transfer, and transformation, with people (or letters or cubes) representing units of energy, hand motions (or letters) depicting energy forms, and regions on the floor (or boxes on paper) representing objects.Energy transfers are represented by, for example, a person moving between regions on the floor, and energy transformations are represented by changing hand motions.Participants in Energy Project PD spend a substantive fraction of their time (i) negotiating Energy Tracking Representations within small groups (usually four to six teachers) and then (ii) presenting their representations to one another.Energy Theater performances or Energy Tracking Diagram presentations often become launching points for larger group discussions about similarities and differences across representations or about questions that the representations raise.
Teachers in Energy Project PD use ETRs to model realworld scenarios, such as raising and lowering a ball, releasing a ball under water, or pushing a block across the floor at constant speed.For example, the teachers in Fig. 1 are performing an Energy Theater for a hand pushing a box across the floor at constant speed.The three regions on the floor (in this case marked by pieces of tape) are the floor (far left group of teachers), the box (middle teacher), and the hand (far right group of teachers).Teachers are using hand signs to indicate thermal energy ("T"), kinetic energy ("K"), and chemical energy ("C").Transformations are indicated when teachers change their hand sign within a single region (e.g., when they change their hand sign from C to K in the hand region), and transformations are indicated when teachers move from one region to the other (e.g., when they maintain their K hand sign while moving from the hand region to the box region).
Facilitation of PD courses.-EnergyProject PD facilitators take a responsive approach to instruction [39][40][41][42][43] in which the curriculum emerges and adapts based on teachers' ideas and interests.The culture is one of collaborative sense-making among teachers, with sufficient time given for the negotiation of ideas and representations [44].For example, Atkins and Frank [45] describe one Energy Project PD course from the summer of 2011, where a group of teachers spent a full week negotiating "five laws of energy."Atkins and Frank show how the substance of the laws evolved as participants debated and then refined the content of their original "possibly true statements about energy," and they illustrate how teachers' questions about particular aspects of these "possibly true statements" drove the conversation.
Links between PD design and research questions.-InEnergy Project PD, teachers apply ETRs to a series of realworld scenarios, becoming more flexible in their use of a theoretical model for energy.We hypothesized that the iterative refinement of teachers' energy model over time, including the process of negotiation and consensus-seeking that happens as teachers create ETRs for various scenarios, may contribute to the development of positive attitudes FIG. 1. Teacher enactment of Energy Theater for a hand pushing a box across the floor at constant speed (reproduced from Ref. [38]).Hand signs indicate forms-e.g., fourth person from the left is making a T (symbolizing thermal energy), person in red striped shirt toward the center is making a K (symbolizing kinetic energy), and rightmost person in black cardigan is making a C (symbolizing chemical energy).
about science.This hypothesis is grounded in research (e.g., Refs.[13,25]) that suggests that courses with a focus on knowledge-building practices in science can lead to shifts in CLASS scores.
Further, our responsive approach to instruction centers on the refinement of participants' ideas in a way that both mirrors community practices in science (e.g., building ideas together) and highlights the centrality of teachers' own thinking in the course.We hypothesized that this responsiveness not only reinforces the epistemological orientation embedded in the process of negotiating teachers' energy model, but also communicates to teachers that they are capable of doing science (e.g., by illustrating that their ideas support collective intellectual progress).Both of these may contribute to the development of positive attitudes about science.
Additional details.-Teachers can participate in up to two summer Energy Project PD courses, and interested teachers continue their learning about energy during the academic year in a biweekly teaching seminar.More extensive descriptions of Energy Project PD have been reported elsewhere (e.g., Refs.[36,38,44,[46][47][48][49]).
Throughout this manuscript, we treat the "Energy Project PD courses" as a single approach, even though the courses are taught by different instructors, and the responsive nature of the courses means that each course takes a somewhat different route.Our primary aim in this paper is to attend to aggregate trends across courses as a coarse measure of effectiveness, as opposed to documenting interesting and significant idiosyncrasies between courses.

B. Sample
During the summer of 2013, thirty-three in-service secondary teachers and fifteen in-service elementary teachers enrolled in Energy Project PD courses.Table I briefly describes the courses and gives (i) the number of teacher participants enrolled in each course and (ii) the number of teachers that completed the Colorado Learning Attitudes about Science survey online both before and after the course.The majority of the teachers in the courses taught in local schools.Our sample represents a range of teaching experience and primary teaching focus (e.g., biology, physics, etc.).

III. RESEARCH METHODS
Overview of CLASS instrument.-Toaddress our research question-"Do teachers express more expertlike attitudes about science after Energy Project professional development than before?"-weadministered the CLASS [12] twice: once in the weeks preceding and once immediately following PD instruction.Participation was mandatory; teachers were required to complete a pre-and post-survey in order to receive a stipend for the course.We chose the CLASS because it is designed to measure beliefs about physics and learning physics.In particular, the survey consists of forty-two test items, organized into eight empirically determined categories: real-world connections, personal interest, sense-making or effort, conceptual connections, applied conceptual understanding, problemsolving general, problem-solving sophistication, and problem-solving confidence.
Using a five-point Likert scale, respondents express the extent to which they agree or disagree with statements that comprise these eight categories.A respondent's score is based on the extent to which their choices on thirty-six of the forty-two items align with the responses of experts.(I.e., Experts are physics faculty "who have extensive experience with teaching [university] introductory physics courses and worked with thousands of students" [12].)This score is expressed as two percentages: a percentage favorable (which expresses the "percentage of responses for which the [respondent] agrees with the experts' view" [12]) and a percentage unfavorable (expressing the percentage of responses for which the respondent disagrees with the experts' view).The creators of the CLASS found that students' interpretations of the degrees of agreement (or disagreement) varied-i.e., that "the same conviction of belief may not result in the same selection such that one student may respond with strongly agree while another responds with agree."For this reason, "strongly agree" and "agree" (two of the choices on the CLASS Likert scale) are treated as the same answer, as are "strongly disagree" and "disagree".
Validity and reliability of CLASS instrument.-Developers of the CLASS validated the survey in an iterative process that involved face validity, construct validity, predictive validity, and concurrent validity.In interviews, students and experts "had consistent interpretations of nearly all of the statements," confirming the "clarity and meaning of [survey] statements."Similar populations perform similarly on the CLASS, establishing its reliability [12].
Measuring shifts in teachers' attitudes using the CLASS.-Using the pre-and post-survey data, we calculated the shifts in percentages of favorable responses from before to after the PD courses-as well as the standard error of those shifts-using the spreadsheet provided by the CLASS developers [50].Because of the small sample size and the nonparametric distribution of our cohort's CLASS scores, we applied the Wilcoxon signed-rank test to our data using the Statistical Package for the Social Sciences (SPSS).The Wilcoxon signed-rank test determines the statistical significance of the differences between average (by course) pre-and post-survey scores (p < 0.05).We measured the effect size, r (i.e., the size of the difference), of the shift in percentages favorable from pre-to post-PD using a method from Rosenthal [51], where r > 0.1 is a small effect, r > 0.3 is a medium effect, and r > 0.5 is a large effect [52].
Given the design and objectives of our course [e.g., (i) our focus on taking up teachers' own ideas, (ii) teachers' analyses of real-world scenarios using ETRs, etc.], we were particularly interested in the extent to which teachers expressed more expertlike attitudes in the real-world connections (RWC), personal interest (PI), and sensemaking (SM) categories.Table II includes the CLASS statements that comprise each of these categories.
Based on the individual items that make up each category, we infer that the CLASS understands learners to be • Making real-world connections (i) when they see that understanding physics helps them to understand the real world, or (ii) vice versa.• Expressing personal interest (i) when they see that understanding physics helps them to understand the real world, or (ii) when they see value and/or feel satisfaction in understanding physics.[There is some overlap between the definitions of RWC and PI, in part because two of the CLASS statements appear in both the RWC and PI categories (see Table II).]• Sense-making (i) when they treat learning and doing physics as effortful, and/or (ii) when they recognize that learning and doing physics is about more than facts and formulas; it also involves sensemaking, conceptual understanding, and reconciling inconsistencies.Each of these represents a more concrete and specific instantiation of our goal to support the development of In physics, it is important for me to make sense out of formulas before I can use them correctly.Spending a lot of time understanding where formulas come from is a waste of time.There are times I solve a physics problem more than one way to help my understanding.When I solve a physics problem, I explicitly think about which physics ideas apply to the problem.When studying physics, I relate the important information to what I already know rather than just memorizing it the way it is presented.
positive attitudes about science among K-12 teachers.In particular, we want our teacher participants to see physics as a useful lens through which to understand the world; we want teachers to become personally invested in their PD experience; and we want teachers to see physics as fundamentally about sense-making, conceptual understanding, and reconciling inconsistencies.

IV. RESULT: TEACHERS EXPRESS MORE EXPERTLIKE ATTITUDES ABOUT SCIENCE AFTER ENERGY PROJECT PD
Our statistical analysis suggests that, overall, teachers express significantly more expertlike attitudes, as measured by the CLASS, after Energy Project PD than before.For all three courses (UE2, E1, and E2), there were significant shifts in overall scores on the CLASS, toward higher percentages of favorable responses, with large effect sizes.
Tables III-V show the breakdown of results by course, listing the mean scores, mean shift in score, and effect size for each CLASS category in the UE2, E1, and E2 courses, respectively.As articulated above, we highlight personal interest, real-world connections, and sense-making, because these are the facets of teachers' "beliefs about physics and learning physics" (or attitudes) that we most wanted to measure, given our goals and course design.
The tables show that for all three courses, teachers express significantly more PI after PD than before, and teachers in UE2 and E1 express significantly more positive attitudes along the RWC dimension.However, shifts in teachers' attitudes along the SM dimension were not significant for any of the courses.
Figure 2 compares the shifts in percentages of favorable responses over the course of Energy Project PD (red bars) to published shifts in percentages of favorable responses over the course of other physics courses (blue), including both research-based and more traditional physics courses.Students enrolled in the courses represented in Fig. 2 include introductory algebra-and calculus-based physics students, nonscience majors, students in upper-division b Effect size calculated using Cohen's benchmarks of large effect for r > 0.5 (indicated by § ), medium effect for r > 0.3, and small for r < 0.3.b Effect size calculated using Cohen's benchmarks of large effect for r > 0.5 (indicated by § ), medium effect for r > 0.3, and small for r < 0.3.physics courses, and preservice elementary teachers.Most courses for preservice teachers are labeled as "Physics by Inquiry," "PET," or "PSET."(For more information, see Ref. [11].)To our knowledge, there is no published record of the CLASS being used with inservice teachers.
We are cautious in interpreting these data as demonstrating the effectiveness and/or relative effectiveness of our professional development approach.Though shifts in teachers' CLASS scores may be a coarse measure of success, we understand that there are many additional factors that may affect teachers' attitudes about science, and we acknowledge that the CLASS captures the expression of these attitudes, rather than the possession of them.However, we are encouraged by the overall consistently positive and relatively high shifts toward more expertlike expressions of attitudes about science.

V. DISCUSSION AND IMPLICATIONS
Teachers who enrolled in Energy Project PD during the summer of 2013 completed the CLASS both before and after instruction, to measure possible shifts in their expressed attitudes about physics and learning physics.We found that teachers in all three courses expressed significantly more expertlike attitudes after PD than before.Table VI summarizes the shifts in teachers' scores along the RWC, PI, and SM dimensions, three categories in which we had a particular interest, given our course design and objectives.
It is beyond the scope of this short paper to attribute these shifts to specific features of our courses.However, we offer some speculations here, drawing on three features of our courses that we highlighted in Sec.II: our use of Energy Tracking Representations to explore and unpack real-world scenarios, teachers' collaborative negotiation and extensive group work, and our responsive approach to instruction.In particular, we hypothesize that teachers' use of representations to understand complex real-world scenarios may contribute to their seeing physics as helping them to understand the real world, and vice versa, corresponding to the CLASS' sense of RWC and PI.Extensive collaboration between teachers and our responsive instructional approach-in which teachers' ideas become a key part of the evolving course material-may further contribute to teachers' PI, since the teachers' own interests and ideas are at the center of their inquiry.Further, the relevance of the particular scenarios-and of energy-to teachers' own classroom instruction may support them in making RWC or enhancing their PI.This latter proposal is consistent with one made by Crouch, Wisittanawat, and Renninger, who suggest that a focus on topics and examples relevant to life-science majors [15] in a physics course for biologists contributed to an improvement in these students' attitudes in science.Teachers' overall improved CLASS scores may be tied to the epistemological underpinnings of the course, in which teachers (i) iteratively refined their model for energy, (ii) used ETRs to analyze real world scenarios, and (iii) experience their own ideas as substantive to further inquiry.That teachers in our E2 course did not post higher scores along the RWC dimension after PD than before may reflect a ceiling effect, since the mean presurvey CLASS score for these teachers was ∼94% favorable.
As reflected in Table VI, we found that shifts in teachers' scores on the SM dimension of the CLASS were not statistically significant for any group.This result surprised us, because we can point to features of our course that we expect to support teachers in coming to see physics as involving sense-making, conceptual understanding, and coherence-seeking (more so than memorizing and applying facts and formulas).For example, we expect that our culture of collaboration and responsiveness-both of which center on the negotiation and refinement of ideas in physics-might contribute to a shift along the sense-making dimension.b Effect size calculated using Cohen's benchmarks of large effect for r > 0.5 (indicated by § ), medium effect for r > 0.3, and small for r < 0.3.However, our evidence does not confirm this hypothesis.One possible limitation is in our use of CLASS to measure this dimension of teacher attitudes.Many of the questions that comprise the sense-making category emphasize problem solving; for our teachers, this may cue thinking about formal problem solving (e.g., computations or word problems), rather than the conceptual problem solving they did in our PD.
As a whole, our work contributes to the larger effort of making the case that PD matters for teacher learning [33],  1 This figure may mislead one to infer that, on average, students in most physics courses post favorable gains on the CLASS.This has historically not been the case-in most typical physics courses, students perform worse on the CLASS after instruction than before [11].Most of the courses in this figure represent reform-oriented or research-based approaches to teaching physics.including teacher attitudes.It also raises questions about the use of the CLASS for measuring in-service teacher attitudes about science, and for measuring shifts for courses with durations of one or two weeks, as opposed to a semester or quarter.With respect to the latter, many in-service teacher PD experiences involve at least as many contact hours as university courses (our courses were 30 and 60 h, respectively).The extent to which this difference in distribution of contact time affects scores on the CLASS, to our knowledge, has not been explored.Future work could investigate whether there are differences in interpretation of the CLASS for in-service teachers, and/or effects on shifts in performance tied to the length of the course.

FIG. 2 .
FIG. 2. Comparison of shifts in percentages of favorable responses over the course of Energy Project professional development (red bars) and shifts in percentages of favorable responses over the course of other physics classes (blue bars).1This figure may mislead one to infer that, on average, students in most physics courses post favorable gains on the CLASS.This has historically not been the case-in most typical physics courses, students perform worse on the CLASS after instruction than before[11].Most of the courses in this figure represent reform-oriented or research-based approaches to teaching physics.

TABLE I .
Information about Energy Project PD courses.

TABLE II .
[12]S statements comprising RWC, PI, and SM categories (reproduced from Ref.[12]).Learning physics changes my ideas about how the world works.Reasoning skills used to understand physics can be helpful to me in my everyday life.The subject of physics has little relation to what I experience in the real world.To understand physics, I sometimes think about my personal experiences and relate them to the topic being analyzed.

TABLE III .
Percentage favorable responses, by category, for second-year elementary teacher PD course (UE2).

TABLE IV .
Percentage favorable responses, by category, for first-year secondary teacher PD course (E1).

TABLE V .
Percentage favorable responses, by category, for returning secondary teacher PD course (E2).