Moving toward change: Institutionalizing reform through implementation of the Learning Assistant model and Open Source Tutorials

Florida International University has undergone a reform in the introductory physics classes by focusing on the laboratory component of these classes. We present results from the secondary implementation of two research-based instructional strategies: the implementation of the Learning Assistant model as developed by the University of Colorado at Boulder and the Open Source Tutorial curriculum developed at the University of Maryland, College Park. We examine the results of the Force Concept Inventory (FCI) for introductory students over five years (n 1⁄4 872) and find that the mean raw gain of students in transformed lab sections was 0.243, while the mean raw gain of the traditional labs was 0.159, with a Cohen’s d effect size of 0.59. Average raw gains on the FCI were 0.243 for Hispanic students and 0.213 for women in the transformed labs, indicating that these reforms are not widening the gaps between underrepresented student groups and majority groups. Our results illustrate how research-based instructional strategies can be successfully implemented in a physics department with minimal department engagement and in a sustainable manner.


I. INTRODUCTION
It has been well established that traditional physics instruction is insufficient for developing the level of physics expertise desired by many instructors of introductory physics [1,2]. When research-based instructional strategies are introduced as an alternative to traditional physics instruction, the difficulties encountered by reformers have included situational characteristics, departmental policies, and the attitudes of the departmental faculty, university administration, and students [3][4][5][6].
In addition to the difficulties faced when initiating course transformation, sustaining transformed courses is notoriously difficult [7]. One path toward sustained course transformation is for the university to institutionalize the transformed course [8,9]. Institutionalization may take on many forms, but we consider a transformation to be institutionalized when the university invests significantly in making the reform the norm through infrastructure, staffing, and/or changes to programs and degrees. With the support of the Physics Teacher Education Coalition (PhysTEC) project, the Physics Education Research (PER) Group at Florida International University (FIU) has undertaken the transformation of the laboratories associated with the introductory physics sequence. These transformed labs are now the standard lab courses in introductory physics. Further, the physics department has incorporated (and funded) Learning Assistants (LAs) into these labs and the university has institutionalized the Teacher-in-Residence (TIR) to oversee the labs and LA program. Collectively, these indicate the degree to which the transformation has been institutionalized. When transformed courses become institutionalized, they have the potential to motivate further reform by influencing a wider selection of faculty and students.
When our Physics Education Research group undertook the transformation of introductory classes, our reforms focused on the laboratories associated with these classes. Choosing to transform the laboratory sections had two rationales. First, faculty do not teach lab sections at FIU and thus would not be required to change their practices. Second, by providing a well-established set of lab activities, equipment lists, and a Teaching Assistant (TA) preparation program, the PER group would be better able to promote and facilitate the institutionalization of the reforms. Additionally, by transforming labs and working with TAs to prepare for the transformed labs, we have been able to introduce over 35 TAs to transformed teaching methods, many of whom will go on to faculty positions. In order to achieve this goal of institutionalizing reform, we chose reforms that were well established and grounded in research, with available curricula and documentation. The two research-based instructional strategies chosen were both secondary implementations of reforms developed at other institutions. The first was the implementation of the Learning Assistant model, where undergraduate students assist in the laboratory instruction as part of an experiential teaching program [10]. The second strategy was the use of Open Source Tutorials (OST), which are worksheets designed to foster small-group conceptual understanding [11,12].
We now describe the reform implementation, present data on improved student conceptual understanding for transformed labs (compared to traditional labs), and provide evidence of the institutionalization of the reforms. We present our case as an example of how research-based instructional strategies can be implemented in a physics department with minimal departmental and administrative resources.

II. REFORMING INTRODUCTORY PHYSICS INSTRUCTION AT FIU
A. Context: An urban Hispanic serving institution FIU, a Hispanic serving institution, awards the largest number of Bachelor's and Master's degrees to Hispanic students in the U.S. [13]. The population of students attending FIU is 60% Hispanic, 15% White, and 12% Black, and 56% are female. FIU is on a large, urban campus, where more than 90% of the 40 455 students commute [14].
Each year, approximately 1200 students enroll in the first-semester physics course during the fall and spring semesters. Most of the classes offered use a traditional lecture style. Five to nine sections of the lecture are offered each semester, ranging in size from 60 to 120 students each.

Learning Assistant model
FIU's physics department implemented the Learning Assistant model [15] as the foundation of its PhysTEC Primary Partner Institute Project [16]. The Learning Assistant model is based on an experiential learning program developed by the University of Colorado at Boulder that recruits and prepares future physics teachers while driving course transformation and institutional change [10]. The LA program in physics at FIU recruits top undergraduate introductory physics students and provides them with the opportunity to engage in hands-on teaching experiences by working in transformed lab courses. In the lab courses, two, or occasionally one, LAs partner with a TA lab instructor. The LAs are not charged with the overall management of the class, but instead are there to help TAs facilitate discussion and interaction, which are markers of the transformed labs.
Preparation of the LAs to work in transformed labs includes two parts. First, LAs participate in a seminar course that couples their teaching experience with the opportunity to explore the intellectual underpinnings of effective instruction. Second, LAs and TAs participate in a weekly preparation meeting during which they complete the lab together, discuss and evaluate prelab assignments, and identify conceptually difficult components of the lab. The sessions are organized and facilitated by the Teacherin-Residence, who not only provides insight into learner resources for the concept at hand but also models questioning and redirection techniques for the LAs. Through the coordinated program, LAs broadly develop their teacher identity by learning about effective pedagogy, improving their content knowledge, and experiencing the rewards of teaching. And, as we will discuss in the results, students in the transformed labs are achieving greater conceptual understanding in labs with the LAs and the Open Source Tutorials.

Open Source Tutorials
The Open Source Tutorials are worksheets designed to foster conceptual understanding, designed by the University of Maryland, College Park, Physics Education Research Group [11,12]. They follow the model of the tutorials developed at the University of Washington, [17] in which small groups of students work collaboratively on worksheets on a single concept while instructors circulate around the room, answering questions and probing students' understanding. The OSTs have an explicit epistemological emphasis, which means students are expected to grapple with understanding their prior knowledge, determine how that knowledge is reasonable, and identify how it fits with the concept being studied. The OSTs are designed so that they can be altered by instructors to fit their particular instructional context. At FIU, students use their three-hour laboratory sections to complete one tutorial and one interactive lecture demonstration (which is completed in small groups in the same format as the tutorial). Students work in groups of three or four, in classes of 30 students, and each class is taught by one graduate TA and one or two LAs. All 13 weeks' topics taught in the first semester have been transformed; 11 use tutorial materials largely based on the OSTs and two were developed by FIU TIRs, TAs, and PER faculty to address topics not covered by the OSTs.
The Open Source Tutorials were chosen as the lab curriculum for both pedagogic and implementation reasons. The pedagogic motivations include their epistemological framework, ability to include LAs in their implementation, and for their coherent pedagogic stance with other reform efforts implemented in the physics department. Implementation barriers were also minimal. OSTs were a fairly comprehensive curriculum, offering tutorials corresponding to 11 of the 13 weeks of first-semester topics. The materials included instructor's guides, which meant that the initial laboratory supervisor did not need to attend offsite training or workshops. Lastly, the open-source format makes it easy to change the materials in response to local needs; these changes primarily consisted of combining each week's tutorial with a related interactive lecture demonstration (also included in the OST package) so that students would have sufficient materials to span each three-hour lab class. A significant additional consideration was that the materials are free, so they could be provided to students for only the cost of photocopies.
The simultaneous implementation of OSTs and the LA program occurred because the two instructional strategies were seen as complementary. Tutorials are usually run with a student-instructor ratio of between 10:1 and 20:1; using LAs allowed us to achieve a ratio of 10:1, even while expanding the number of students in each TA-led section from 24 to 30. Likewise, OSTs provided an interactivelearning environment where LAs could put effective pedagogies into action and experience the rewards of teaching. In addition, although we have not explored OST use without LAs, our sense is that LAs have enhanced the effect of the OSTs by facilitating conceptual reasoning and epistemologically focused discussion during the lab. The complementary theoretical framework of these two reforms will be further discussed in Sec. IV.

C. Navigating the institutional landscape
To provide effective teaching models for the LAs, the LA program uses research-validated curricula; thus the program provides an impetus for course reform. In this case, the target of reform was the introductory physics laboratory, as it provided a means to effect positive change on all introductory physics students with minimal barriers to implementation. Labs are ideal spaces for implementing informal cooperative group learning environments where LAs can play an important role. The three-hour allotted lab length allows time for deeper engagement and development of conceptual understanding. Almost all introductory physics students must take lab, thus providing a mechanism to impact learning across the introductory course sequence. Further, lab reform is generally not discouraged by physics faculty, allowing the reform efforts to be directed at implementation and not at conflict resolution.
Providing evidence on the effectiveness of the reform implementation is vital both to the LAs and to the lab reform program; therefore, a research component is integrated. Research data, such as those included in this paper, provide feedback to the Learning Assistants and inform their instructional practice. The research component also appeals to the science identity of LAs, essentially all of whom come from science, mathematics, or engineering backgrounds. For the lab transformation project, the research provides feedback to guide the reform implementation. Beyond the lab reform project, the data can provide compelling evidence that encourages expansion of the reform efforts across the university. This has also led to other science and mathematics disciplines implementing LA programs in their departments, often as a result of their students advocating for change. Thus, the LA program provides a pathway to institutional change.

III. LITERATURE
In response to the demonstrated need for improved introductory physics instruction, the field of PER has produced many types of research-based strategies [17][18][19][20]. In spite of extensive dissemination efforts, these strategies have not been adopted by a significant number of university physics instructors [4].
Research indicates that the norms at universities do not support many of the changes called for by those seeking to improve undergraduate education [3,5,6]. In particular, Prosser and Trigwell [21] found that university instructors' adoption of student-focused instruction was influenced by many factors, including (1) instructors' perceptions of how much control they had over how their class was taught and (2) their perceptions of how much their department valued teaching. Other significant barriers to using research-based instructional strategies include expectations of content coverage and student resistance [4,22]. There are also indications that the relationship between physics professors and members of the Physics Education Research community can hinder adoption of reforms [23].
This research points to the difficulties faced by an individual seeking to implement research-based instructional strategies in physics, whether at the level of her classroom or at the departmental level. The strategy adopted at FIU of transforming the laboratories rather than lectures was in part an effort to avoid possible faculty or departmental resistance; because laboratories are led by graduate Teaching Assistants, most faculty are not involved with this part of the introductory physics instruction.

IV. THEORETICAL FRAMEWORK: SUPPORTING EFFECTIVE EPISTEMOLOGICAL STANCES IN STUDENTS
When students come into a university classroom, they arrive with many years of educational experiences that help shape their expectations of the types of activities in which they anticipate participating. These activities might include listening to the professor lecture, taking notes on what he says or writes on the board, or raising their hands before asking a question. Students also have expectations about what they will need to do in a class in order to generate answers acceptable to their teachers. Students' views of how they should go about learning are their epistemological beliefs [24]. These often tacit beliefs are context dependent; the same person who supports a physics claim with ''the professor told me so'' (perhaps treating knowledge as ''propagated stuff'') might also devise a creative solution in a computer lab (treating knowledge as ''fabricated stuff'') [25]. This example also illustrates the idea that epistemological beliefs are neither right nor wrong, but rather appropriate (or inappropriate) for a given context.
In physics classrooms, instructors frequently encounter students with counterproductive expectations, such as the idea that physics is mostly about ''finding the right equation to use'' ( [26], p. 219). Research shows that when students experience changes in their environment and activities (such as using collaborative worksheets with open-ended questions instead of recipe-book laboratories), this alters how they approach knowledge [27,28]. Thus, the choice of instructional strategies for FIU's introductory physics labs was guided by a desire to provide a context that encouraged students to activate productive epistemological beliefs.
OSTs consist of questions and activities sequenced specifically to engage students in discussions of their own understanding of the phenomena and to coordinate that understanding with accepted scientific knowledge. This activity is different from a setting that encourages students to receive knowledge. Thus, OSTs are designed so that students can experience knowledge creation in a way that is more aligned with how scientists think. They also prompt students to explicitly consider what epistemological beliefs they are using in physics labs and whether these are the most appropriate ones.
The use of Learning Assistants in the labs furthers the goal of encouraging the use of appropriate epistemological beliefs. First, undergraduates who apply to participate in the LA program have an interest in learning and helping others learn. Virtually all FIU physics LAs have experience with constructivist learning, either through participating in one of the reformed labs or in a fully reformed class. These experiences provide them with tools and attitudes that are more aligned with productive epistemological beliefs. Additionally, the seminar course that LAs take at the start of the program prompts them to focus on students' epistemological beliefs. We chose to implement OSTs and the LA model because together they help support the epistemological stances we want introductory physics students to employ when learning in their labs.

A. Population
In order to investigate the impact of the transformed introductory physics labs, we collected Force Concept Inventory (FCI) data. Students from whom we collected data were taking introductory calculus-based and algebrabased physics courses. The students take the FCI in the lecture component of the course. Lab sections are not associated with any particular lecture section; thus, students may sign up for lab sections based on their schedules and lab availability. This allows us to examine the effect of the laboratories, as students are distributed evenly across the lecture sections.
For the purposes of this paper, we exclude two distinct groups of students. The first group is students enrolled in fully transformed courses taught by PER faculty. Approximately 15% (373 of the 1633) of the overall group of students in introductory physics take integrated lab and lecture courses taught by physics education research faculty who primarily use modeling instruction [29] and ISLE [30]. These classes have teaching methods and learning gains that are significantly different [31] and they do not use Open Source Tutorials; thus, we cannot make comparisons of the effects of both reforms on students in the fully transformed classes and the students in transformed labs. From the remaining group of 1260 students, we exclude data from students who did not complete the laboratory during the same semester as their lecture. Some majors do not require students to take the laboratory, and some students take the lab asynchronously from the lecture due to scheduling and/or failure. Because we could not separate those students who had previously taken the laboratory from those who had not yet taken it, data from this group could not be used to understand the effect of the transformed labs. As a result, we excluded 387 of the remaining 1260 introductory students (approximately 24% of our original group of 1633 students) from whom we collected data because they did not enroll in a laboratory concurrently with their lecture. From the remaining 873 students, one additional student was excluded because he achieved perfect scores on both the pretest and the posttest. As a result, data from 872 students are included in this analysis.
Students enrolled in both the calculus-based and algebra-based courses take the same laboratory component. Because we administer the FCI during the lecture, and because most of the lecture instructors who participated were teaching calculus-based courses, the majority of our data (80%) is composed of students taking calculusbased courses. While we do not present the results in this paper, raw gains achieved by algebra-based students in transformed labs were comparable to the raw gains achieved by the larger student population in the transformed labs.
The students discussed in this paper are undergraduate students enrolled in first-semester physics from Fall 2005 to Spring 2010. Of the 872 students whose data make up this study, 493 students were enrolled in traditional labs and 379 were enrolled in transformed labs. The total population was composed of 552 men (63% of the total students) and 320 women (37% of the total students). As reported in student records, the distribution of ethnicities of the total population was as follows: 625 Hispanic (72%), 100 White (11%), 71 Asian (8%), 64 Black (7%), and 9 unreported (1%), which is reasonably aligned with the population of FIU. For this paper, we analyze differences between treatments and then disaggregate the data to compare the effects of the lab by gender and by ethnicity. These comparisons are important to determine if the effects of the lab are equitably realized in our diverse student population.

B. Measures: Raw gain on the FCI and standardized effect sizes
We measured the students' conceptual understanding using the FCI, a paper-and-pencil test composed of 30 conceptual questions. The FCI has been used extensively to assess the conceptual understanding of students in many instructional environments [1,18,19,32]. The FCI is administered as a pre and post assessment. We use a strict matching requirement that students must have completed both the pre and post FCI to be included in the data set.
We argue that, when comparing student populations with different initial knowledge levels, using normalized gains obscures the gains achieved by underrepresented minorities [33]. For instance, in a case where women begin instruction with a lower FCI score, if they increase their scores at the same rate as men they would still have a lower normalized gain score. For this reason, and the fact that hgi does not include any variance estimates, normalized gain is not particularly useful when looking at subgroups within a single treatment. Instead we advocate using effect sizes on the raw gain to compare different treatments. Thus, we report the pre FCI, post FCI, raw gain (post%-pre%), effect size on the raw gain, and confidence interval (C.I.) on the effect size for all student subgroups (examined by gender and ethnicity). In this study, we utilize Cohen's d as the effect size to compare traditional labs with transformed labs. For further discussion of effect sizes and confidence intervals, see [34][35][36][37].

VI. INTRODUCTORY PHYSICS STUDENTS' IMPROVED CONCEPTUAL UNDERSTANDING
There is a significant difference in the raw gains of the students taking transformed labs compared to those taking traditional labs. The mean raw gain of the transformed classes was 0.243 and the mean raw gain of the traditional classes was 0.159. When comparing the raw gains of the traditional to the transformed labs, there is a Cohen's d effect size of 0.59 with a 95% confidence interval of (0.45, 0.72). We believe, using Cohen's d guidelines, that this C.I. and effect size indicate a moderate to large shift in FCI scores ( [38], p. 192).
In order to compare the effect of different implementations, we compare the pretest to posttest effect sizes. Thus, we compare the pretest to posttest effect size, seen in Eq. (1), of both FIU courses to those from other physics courses (both interactive engagement and traditional).
In Fig. 1, we show effect sizes and the 95% confidence interval calculated for the university data presented by Hake [1] and we include effect sizes for FIU transformed and traditional labs. The effect size of taking a class with a transformed lab is 1.32, with a 95% confidence interval of (1.16, 1.48), while the effect size of taking a class with a traditional lab is 0.91, with a 95% confidence interval of (0.78, 1.04). As the confidence intervals on the effect sizes of the FIU transformed labs and FIU traditional labs do not overlap, we are 95% confident that the effect of a transformed lab at FIU is larger than the effect of a traditional lab. Furthermore, the confidence interval on the effect size of the FIU transformed labs is higher than four out of the five confidence intervals of the traditional university courses reported by Hake. The effect sizes of the FIU traditional labs are comparable to the effect sizes of the traditional university courses reported by Hake.
The demographics at FIU, a Hispanic serving institution, provide us with the opportunity and obligation to disaggregate the data by student subgroups, examined by gender and ethnicity. Table I summarizes the pre FCI, post FCI, raw gain, effect size, and confidence interval on the effect size for all student subgroups. Each comparison includes the standard deviation (SD) for each mean value and Cohen's d effect size and 95% C.I.
When comparing the effect sizes of transformed versus traditional labs, we see no differences between the gains experienced by women as compared with those for men. The effect size for women was 0.59, with a confidence interval of (0.36, 0.81), and the effect size for men was 0.60, with a confidence interval of (0.43, 0.77). Because these confidence intervals overlap, we cannot say that there is any difference in the effects experienced by women versus men. We find similar results when comparing subgroups disaggregated by ethnicity. The confidence intervals on the effect sizes of transformed labs' gains for Hispanic students (0.37, 0.69), White students (0.40, 1.23), Black students (0.12, 1.15), and Asian students (0.14, 1.15) have extensive overlap; therefore, we cannot see differences in the effects experienced by the four ethnic subgroups. Our data indicate that the transformed labs are not preferencing either gender or ethnic subgroups. However, it should be noted that the Hispanic students in the two treatments did start out with different FCI scores (transformed started higher), as indicated by the confidence interval on the effect of grouping not including zero. It is possible that the preinstruction differences are one source FIG. 1 (color online). A comparison of Cohen's d standardized effect sizes on raw gain on the Force Concept Inventory from various instructional strategies. The data include effect sizes of interactive-engagement courses reported by Hake [1] in red and effect sizes of traditional courses reported by Hake in blue; the categories were determined by Hake and only university data are included. We report FIU results for transformed labs (shaded red) and traditional labs (shaded blue). Error bars represent 95% confidence intervals on the effect size.
of differences among Hispanic students, but the effect of grouping within Hispanic men is small by Cohen's guidelines. These reforms are not closing gaps that exist between underrepresented groups and majority groups, but neither are they widening the gaps.

A. Explaining successful reform
From the data presented in the previous section, we see increased FCI scores, which indicate increased conceptual understanding in students completing transformed labs compared to those students who completed the traditional labs. These results are noteworthy for two reasons. First, the transformations were implemented in the labs, though the effect is seen through the FCI, which was administered in the lectures. Second, although the effect sizes from the lab were not as large as those seen in many interactive engagement classes, our reported results are noteworthy because the transformations were limited to the laboratories. We would characterize the lectures as traditional, consistent with Hake's definition of classes in which instructors rely ''primarily on passive-student lectures, recipe labs, and algorithmic-problem exams'' ( [1], p. 65).
As seen in Fig. 1, the effect size on the raw FCI gain in FIU's transformed labs is greater than the effect sizes of four out of the five traditional university courses reported by Hake, so we consider the transformations at FIU to be a minimum step toward reform. Moreover, the transformed labs are not widening the gaps between underrepresented minority groups and majority groups, as shown by the lack of evident differences between the gains of men and women and between the gains of the four ethnic subgroups. Without initially implementing substantial changes at the departmental or institutional level and without altering lecture classes, we were able to achieve conceptual gains. It is difficult to precisely assess how much the various instructors (faculty, TAs, and LAs) support the transformed labs. There appears to be little evidence of either overt buy-in or resistance from faculty at this time: there is no coordination between the lecture instructors and faculty member supervising the labs, the lectures and labs have separate homework, and students receive separate grades for each component. We have not attempted to formally assess the buy-in of FIU TAs; previous research on TAs' teaching tutorials suggests that it is difficult to informally assess TAs' support of the curriculum [39]. However, there is evidence of buy-in from LAs, because 75% of all LAs (over all years of the LA program) choose to participate in the program a second time, where about half of physics LAs are assigned each semester to teach OSTs.
While many factors can influence the success of department-wide reforms, there are two particular factors that have supported these reforms: the secondary implementation of previously developed research-based instructional techniques and setting a minimal barrier necessary for departmental support. The secondary implementation allowed us to draw on reform techniques already developed and tested at the University of Colorado at Boulder (the LA model) and the University of Maryland, College Park (OSTs) and focus on the implementation. The reforms were implemented in the discussion sections taught by TAs rather than faculty, thus only minimal departmental and faculty support was necessary at the onset of the transformation process. Once the implementation showed evidence of success, through improved conceptual understanding and other indicators, the department's support increased, as we will discuss in greater detail in Sec. VII B.
We also note that the responsibility for the reforms was undertaken by our Physics Education Research group. We cannot establish the influence of the group on the overall success of the project, other than to indicate that it is possible to establish department-wide reform under the leadership of a PER group. We do, however, suspect that reforms can be established under the leadership of a dedicated team of faculty.

B. Significant institutionalization
The reform efforts undertaken by the physics department have been institutionalized at FIU, building a sustainable foundation for furthering reform in the sciences and mathematics. At the department level, institutionalization is evidenced by the fact that the physics department has provided funding for the undergraduate LAs since 2009; it is expected to continue into the foreseeable future. We note that the departmental LA funding was voluntary and motivated by the value associated with the program. The OSTs are deployed in all 50 sections of introductory Physics I labs, serving over 1200 students each year. The lab TA training program is now part of the graduate teaching seminar required of all graduate TAs. Furthermore, the Colleges of Arts & Sciences and Education have jointly assumed funding of the Teacher-in-Residence position.
On a larger scale, the program's impact has prompted replication of the LA program in the Chemistry, Earth & Environment, and Mathematics Departments, with Biology participation planned for Fall 2011. The LA program has been incorporated into the newly revised secondary teacher preparation programs in Physics, Chemistry, Earth & Environment, and Mathematics (with Biology planned for Fall 2011). These efforts have led FIU to join the Science and Mathematics Teacher Imperative (SMTI) [40]. SMTI is the largest new science and mathematics teacher initiative in the nation, operated by the Association of Public and Land-grant Universities. SMTI subsequently invited the FIU Arts & Sciences Dean to serve on The Leadership Collaborative and our President to serve on their Executive Committee, thus showcasing FIU's programs on a national scale. Thus, the Deans of the Colleges of Arts & Sciences and Education, the Provost, and the President are committed to the LA program, as well as to science and mathematics education reform in general.
The expansion of the LA model across these disciplines impacts the long-term institutionalization of LAs in the physics department. Broad implementation drives enculturation of the LA model within faculty norms and values as well as necessitates the developing administrative structures and procedures to provide support. The number of students serving as Learning Assistants has grown dramatically in the past seven semesters, as shown in Fig. 2 sections of seven different physics courses plus an additional 45 LAs supporting chemistry, mathematics, and earth sciences courses. This includes roughly 40 new LAs in the four departments in each of the past two semesters. Figure 2 also shows the propagation of the LA program across the science and mathematics departments. (Note that the adoption rate is similar in three out of four departments.) Mathematics was the second department to adopt the LA model, growing to roughly 30 LAs in the past two semesters. Chemistry and Earth Sciences have also added LAs to their instructional program in the past 15 months. Evidence of departmental buy-in is shown through the funding of salaries for the Chemistry and Mathematics LAs being provided by internal departmental funding.
The trajectory of institutionalization at FIU is similar to that of the University of Colorado at Boulder, the developer of the Learning Assistant program. At the University of Colorado, the elements leading to program institutionalization included departmental advocacy for smaller tutorial sections, partial departmental and university funding for the program, and non-PER faculty supervising the weekly tutorial training meetings [10,41,42].

VIII. CONCLUSION
Models for sustainable implementation of reform are highly sought after, especially in educational settings involving substantial populations of historically underrepresented students in physics. Our effort to reform the introductory physics labs at FIU was leveraged off the successes of the Learning Assistant model and the Open Source Tutorial curriculum project. By utilizing externally developed models and curriculum, we reduced the burdens that regularly impede course transformation. The onus of developing a full lab curriculum was not on us as we were able to adapt materials that were externally developed. Furthermore, we were able to utilize the materials and approaches for developing skilled discussion facilitators by using the Learning Assistant model. We elected to focus our efforts in the labs, which alleviated the need to change the practices of faculty, who are often reluctant to undergo the course transformation process. This attention to the labs has the added benefit of engaging an entire cohort of future physics faculty and providing them with the experience of using reformed practices. Finally, though our efforts targeted labs, the benefits were not seen merely in the labs. Instead, we have documented increased conceptual understanding by students in the lecture, and these benefits are achieved equitably across both gender and ethnic subgroups.
Considering what has driven the successful implementation of a sustainable physics instructional transformation, it is clear that multiple factors contributed. First, selecting and implementing well-developed, research-validated methods allowed the Physics Education Research group to wholly focus on the implementation of the changes. The use of research-validated methods provided us with more accurate expectations of achievable results and a cohort of colleagues whom we could consult. We also carefully examined the larger physics instructional atmosphere at FIU before setting off on the transformation, so that we could determine where to devote our efforts for maximum impact. Ultimately, we matched our strategies within the larger context of the university setting, essentially using our analytical physicist skills to maximize the benefit and buy-in of the university. Finally, by researching the changes in conceptual understanding, we provided evidence of the efficacy of the paired reforms, which, in turn, has led to further implementation across the university.