Enabling and challenging factors in institutional reform: The case of SCALE-UP

Local success, administrative support, interaction with outside users and funding are common features of successful implementations of SCALE-UP.


I. INTRODUCTION
Policy makers, researchers, higher education administrators, and faculty have tried to improve undergraduate STEM education for many years but progress has been frustratingly slow. There are significant concerns that outdated and inappropriate instructional practices may threaten global scientific technological learning and progress [1][2][3]. Higher education experts have advocated for a paradigm shift from teacher-centered to studentcentered instruction [4,5] and many new instructional strategies have been developed and tested [6,7]. Yet these significant research and development efforts have only moderately changed instructional practices [8]. Many important organizations are concerned about the slow incorporation of research-based teaching ideas into undergraduate STEM instruction [9][10][11][12][13].
Since research-based instructional strategies will only have a significant impact if they can be successfully implemented at nondeveloping sites, it is important to understand how and why secondary implementations succeed or fail. Most of the available literature on research-based instructional strategies comes from the "best-case scenario" of development sites that have access to grant funding, a project team, faculty release time, and education experts that contribute to success. Very few studies examine how secondary sites initiate, implement, and use the products of educational research. Often developers do not articulate the underlying organizational Published by the American Physical Society under the terms of the Creative Commons Attribution 3.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI. support structures needed to ensure successful outcomes and the literature rarely explicitly advises secondary sites on how to overcome challenges and achieve success in different environments. This can contribute to disappointing results when secondary sites try to use the same reform with different students, instructors, and educational settings [14][15][16][17]. To identify why research-based educational transformations do not spread as well as expected, one must open the mysterious "black box" of the implementation stage and share this knowledge with not only consumers, but also developers of physics education research (PER) [18].
Understanding what problems secondary users encounter when implementing new instructional strategies is important because lack of this knowledge-by both developers and potential adopters-can result in failed adoptions. For example, Henderson and Dancy [8] conducted an exploratory study with five physics instructors who were not part of the PER community. They found instructors were aware of problems with their own teaching and familiar with ideas about how to improve. However, these instructors encountered a variety of situational challenges (including student attitudes, expectations of content coverage, lack of instructor time, departmental norms, class size, etc.) that promote the use of more traditional pedagogies [18]. The researchers postulated that situational challenges could be major impediments to reforming teaching and advocated for follow-up studies to document and investigate challenges.
In a follow-up survey of 722 physics faculty [19], these researchers found while a large number of instructors tried research-based instructional strategies, approximately onethird had abandoned the strategy at the time of the survey. This represented the biggest loss point in the adoption process and clarified further the need to understand the challenges of secondary implementations.
The work reported in this paper investigates lower than desired spread of research-based innovations by studying the initiation of 21 successful implementations of a particular radical instructional strategy: Student Centered Active Learning Environment with Upside-down Pedagogies (SCALE-UP). As will be discussed in more detail, SCALE-UP modifies typical pedagogy and classroom design to promote student-centered instruction, active learning, and interactivity between students and instructors. It is also associated with highly positive outcomes [20]. We investigated how individual, departmental, and institutional structures promote or impede sustained use of a SCALE-UP style instruction in higher education STEM departments. This work provides direct relevant advice to faculty, administrators, and others interested in initiating SCALE-UP instruction, especially those who might have run into difficulties along the way. This work also has implications for people interested in implementing other radical reforms that involve structural changes.

A. Goals of the study
In prior work, we used a web survey to develop a census of SCALE-UP users in the United States [21]. The survey results provided evidence that some secondary sites achieved successful, sustained implementations. For example, they had been using the reform for an extended amount of time or the reform spread to multiple departments. The goal of this study is to use interviews to understand what helped or hindered implementation of SCALE-UP at 21 successful sites. By focusing on successful implementations, we are able to provide advice, information, and tools to change agents at other potential implementation sites. Furthermore, this work is of interest to developers of reformed instruction who want their reforms to achieve success outside the development sites. This study will help developers of innovative teaching strategies anticipate potential challenges secondary sites may face and suggest support and advice they can provide to aid effective spread.

B. Research questions
This study is guided by the following research questions.
(1) What factors enable or challenge the initiation of successful SCALE-UP implementations? (2) What factors enable or challenge the sustained use and spread of successful SCALE-UP implementations? (3) How did stakeholders in these successful implementations take advantage of the enabling factors and reduce the impact of the challenging factors?

C. Introducing the SCALE-UP reform
In the early 1990s, Jack Wilson at Rensselaer Polytechnic pioneered "studio physics." They list the defining characteristics as "integrated lecture/laboratory form, a reduction in lecture time, a technology-enhanced learning environment, collaborative group work, and a high level of faculty-student interaction" [22]. Robert Beichner adjusted this studio-style instruction for his large enrollment classes at North Carolina State University, giving it the acronym SCALE-UP "Student Centered Active Learning with Upside-down Pedagogies." This mode of studio-style instruction has successfully spread to a wide range of class sizes, institution types, and dozens of disciplines at over 300 higher education departments worldwide [21]. In this study, we use SCALE-UP as a case study to examine the enabling and challenging factors of institutional and departmental-level change.
We selected SCALE-UP because of its extensive spread, more than most research-based instructional strategies, and because it is a radical reform in terms of both pedagogy and structure. Radical reforms have greater promise to improve student outcomes than more modest reforms [7], and SCALE-UP has been documented to improve student conceptual understanding, problem solving, attitudes toward science, and drastically cut failure rates, especially for women and other underrepresented populations [20]. SCALE-UP radically changes the pedagogy and classroom structure to promote interaction in a studio setting. The studio classroom looks more like a restaurant than a traditional lecture hall. Students are seated in teams at special tables (either round, D-shaped, etc.) designed to promote face-to-face interaction. Student whiteboards serve as public thinking spaces. Multiple LCD or projection screens indicate there is no "front of the classroom" and allow student work to be showcased. The original SCALE-UP model for introductory physics integrated the laboratory and lecture sections of the course so students would have consistency with their same instructors. The instructor minimizes lecture in favor of real-world problem solving, computer programming, and laboratory activities. SCALE-UP replaces the traditional separation of a science course into lecture, recitation, and laboratory by integrating these components so the same students and instructors work together for 4-6 hours per week in a studio-style classroom.

D. Selecting a change model
One of the most widely used change strategies for structural reforms is Kotter's 8-Step Change Model [23][24][25]. Kotter designed this model to help businesses adapt to new, more challenging market environments [23]. Although developed in a business setting, Kotter's model has also shown itself to be useful in understanding organizational change in educational settings [26,27].
Kotter updated his 8-step process in his 2014 book Accelerate [28] to serve different context and objectives for modern society. For example, his older models [23,24] focus on driving change with a small, powerful core group functioning within a traditional hierarchy, affecting episodic change in finite ways to improve one thing in a linear fashion over time. His updated model runs by empowering people across the organization in a network outside a traditional hierarchy, running steps concurrently and continuously, and constantly seeking new initiatives [28]. While some modern businesses might be able to escape a traditional structure, educational reform still largely operates within a hierarchical structure. Thus, his older model is a better fit for analyzing educational reform, and has been used to understand this context before [26,27]. Specifically, Graham [29] argues that Kotter's 1996 model of top-down change has been the most influential in engineering education, perhaps because Froyd, Penberthy, and Watson [30] provided an adaptation of the steps to engineering undergraduate education change initiatives.

E. Introducing Kotter's 8-Step Change Model
Kotter's steps provide a useful point of comparison for understanding the change process involved in implementing SCALE-UP. In this section, we briefly introduce Kotter's model and will use his model to support our recommendations. Kotter's steps can be broadly grouped in three categories: (i) create a climate for change, (ii) engage and enable the organization, and (3) implement and sustain the change.
Before we present the data, we will introduce the basics of Kotter's 8-Step Change Model [23,24] and related results from the higher education literature. Relevant recommendations will be brought in throughout the analysis.

Creating a climate for change
(1) Establish a sense of urgency.-Kotter believes that people will not change if they cannot see a reason to change. Kotter recommends that leaders undertake bold and risky actions to create this need and convince the organization that business as usual is unacceptable. In business situations, examples include a potential revenue drop when a patent expires, declining margins in a core business, or an emerging market that no one sees [23].
When presenting instructors with a curricular innovation in education, faculty need to be convinced change is worth the time and effort. Thus investing time in establishing a need for change will reap significant benefits in implementing subsequent steps.
(2) Create a guiding coalition.-In order to support the change agent, the organization must assemble a group with power, energy, and influence to lead the effort. Kotter [23] acknowledges no one person is capable of single-handedly leading and managing the change process. Thus, the right "guiding coalition" of people is critical to its success and should include people with power, expertise, credibility, and leadership. In higher education, involving a supportive, respected chair or head with authority as well as senior and junior faculty facilitates change [31]. It is also recommended to include some faculty and staff who are "open-minded skeptics" who challenge the need for innovation and the process proposed to achieve the innovation.
(3) Develop a vision and strategy.-The first task of the guiding coalition is to formulate a "clear and sensible vision" for the transformation effort [23] and subsequent plan. This step includes developing strategic initiatives to achieve that vision.
In education, this may take the form of articulating learning objectives or even broader goals for the reform. Froyd, Penberthy, and Watson [30] believe articulating values is key to achieving successful change, but missing from most current change models. Many people try to achieve educational change by conducting studies documenting the superiority of their new method. These educational experiments can only measure if a given reform matches up to some predetermined metric, not dictate the value of intended outcomes. Thus, it is up to practitioners (instructors and administrators) to set values and priorities with regards to outcomes, such as particular learning objectives and attitudinal goals. This can reduce uncertainty [32], decrease ambiguity, and can even affect the type of positive or negative responses to organizational change [33]; Kotter insists "two-way communication is always more powerful than one-way communication" [23]. Raising a large force of people who are ready, willing, and motivated to drive change can also communicate the change vision. In higher education, Chasteen et al. [34] discuss the importance of developing and widely communicating a vision. They recommend, "Get key people involved early and 'pre-sell' the idea to them; plan how future instructors will be introduced to the goals, materials and expectations of the course."

Engage and enable the organization
(5) Empower broad-based action.-Stakeholders need to be empowered to take action. Kotter recommends involving many people in the change effort and getting them to think productively and positively about the change [23]. Empowering the organization often involves overcoming four common obstacles including structures, skills, systems, and supervisors [35].
Wieman et al. [31] give relevant examples of empowering broad-based action in higher education science departments. First, they emphasize the importance of broadly targeting the department as the unit of change. Since the department typically decides what is taught and how it is taught, any sustained attempts to change teaching practices must focus on changing this culture. Wieman et al. recommend that reforms should affect most of the undergraduate courses and most faculty members to create this cultural shift. To engage a wide population, the institution started their change process by putting out a call for competitive collaborative proposals from departments, which led to discussions about instructional goals and, eventually, instructional changes amongst members of the department.
(6) Generate short-term wins.-Acknowledging progress toward change is critical for maintaining momentum. Short-term wins demonstrate that the change effort is paying off and help guide modifications as the vision is put into practice [24]. This involves consistently producing, tracking, evaluating, and celebrating small and large accomplishments and correlating them to results.
In higher education, formative evaluation should be conducted to maximize the potential of change agents by using real-time feedback to adjust program elements based on what works. Short-term wins can take the form of positive results from the initial trial of new instructional strategy. Research collected outside their institution and/or outside their discipline is seldom enough to change an instructor's pedagogy [29]. Skeptical instructors are more convinced by data on student learning from their own courses [29], which emphasizes the importance of documenting local progress.
3. Implement and sustain the change (7) Consolidate gains and produce more change.-Reformers need to create momentum for change by building on early successes to invigorate further action. Since new processes can regress, short-term gains should not be the endpoint of reform: change agents must use these to monitor and continue to tackle other issues, such as systems and structures that are not in line with the recently implemented changes [23]. Increasing credibility can be used to change systems, structures, and policies that do not align with the vision. Reformers should hire, promote, and develop employees that aid the implementation effort and reinvigorate the process by bringing in new projects and volunteers.
(8) Anchor new approaches in the corporate culture.-In order to institutionalize the change and ensure long-term success, Kotter [23] believes that new behaviors are subject to degradation if they are not rooted in social norms and shared values. He lists two specific factors that can help permanently integrate the change in organizational culture: (i) showing how the new approaches, behaviors, and attitudes help improve performance and (ii) ensuring that the next generation personifies the new approach [23]. Reformers are encouraged to articulate how the new behaviors are linked to organizational success and develop the means to ensure continued use through leadership, development, and succession.
In education, reforming instruction requires an environment supportive of change as well as a culture that values learning about teaching [36]. Culture is shaped by such characteristics as administrators' and faculty's values and beliefs, the dominant teaching style, the emphasis placed on teaching, and the willingness of the educational community to discuss teaching and learning [37]. Building or changing a culture does not just come from the top or the bottom but from all sides. Recommendations for creating an amiable culture in education includes regularly discussing how student learning relates to the institutional mission, promoting conversations about research-based teaching, allowing instructors time for innovation, and providing incentives for faculty to participate in professional development [38].

II. METHODS
The goal of this study is to better understand the factors that lead to successful secondary implementations of radical pedagogical reforms. We do this through an analysis of 21 successful implementations of the SCALE-UP instructional strategy. Our data come from telephone interviews with a key informant for each implementation. Our analysis focuses on an emergent analysis of enablers and challenges to each change effort. We then compare the results of this emergent analysis to the recommended practices for institutional change from Kotter.

A. Sampling
A web survey and a census of SCALE-UP users was a precursor to this study [21]. In fall 2013, survey invitations were sent to the approximately 1300 members of the developer's SCALE-UP database. This database includes people who signed up for access to the website to receive more information about SCALE-UP, and many of whom attended a talk or workshop by the SCALE-UP developer. Thus, respondents included past, potential, and current users of the reform, as well as people who were influenced by the ideas but do not replicate the reform exactly. The final sample included 639 survey respondents (a 49% response rate). Using the survey data, we conducted several levels of analysis [21], including a social network analysis [38].
Using the survey and the social network analysis, we used the following criteria to identify the 21 successful implementations for this study: (1) We only included sites in the United States to eliminate interpretation difficulties due to differences in higher education systems in different countries. These differences might obscure important phenomena that may be more visible within a single cultural context. In addition, the majority of survey respondents were from the United States, the country where the reform was developed. (2) We chose secondary sites that modified both the classroom and the pedagogy in line with SCALE-UP recommendations. All of the chosen sites had adopted more interactive pedagogies and, most importantly, have access to a dedicated studio-style classroom. (3) We restricted our sample to "successful" implementations in higher education STEM settings. We defined success with a variety of metrics and chose sites that fell into at least one of the following categories: high spread across university (which reported at least 5 departments using SCALE-UP), the most influential implementations in spreading information about SCALE-UP according to the social network analysis, [38] high longevity (reform use for more than six years) and high penetration in a department (over 60% of introductory courses taught using SCALE-UP). We sent out interview requests to approximately five departments in each of these categories (picked at random from the available list when more than five were available, for example, in the high longevity category). We reached out to people who were listed as key departmental or institutional contact people by survey respondents. These "key initiators" were faculty, department heads, administrators, or professional development staff who were especially knowledgeable about the local SCALE-UP initiation. When people declined interviews, we supplemented our initial sample with similar sites with similar characteristics.
Our unit of analysis was independent, uncoordinated implementation efforts. Some implementations of SCALE-UP were a top-down, coordinated effort initiated by the administration. Even if it affected multiple departments, it would be considered an independent implementation effort because of overarching coordination and planning. In other cases, SCALE-UP was initiated in a bottom-up manner within a single academic department. In these cases the key informant was often the faculty champion behind the SCALE-UP effort. We also considered this an independent effort. Institutions with only one implementation effort will be uniquely identified with their institution letter (for example, interviewee from Institution A).
At two institutions, SCALE-UP efforts started as independent, uncoordinated efforts within two separate departments. In such cases, we interviewed key contact people in both departments because they each had a unique initiation story and this would count as two independent implementation efforts. For the two institutions that have two separate implementation efforts, the interviewee will be distinguished with a letter for the institution and a number (for example, instructor 1 from Institution B or instructor B1).
We requested interviews with 63 people, conducted pilot interviews with five to refine our interview protocol, and conducted full interviews with 31 key contact people who were willing and available to share their experiences. We removed 10 from the sample because the information revealed that they do not fit our criteria of successful, studio-style reforms with dedicated classrooms. The 21 interviews we retained represent 21 separate reform initiation efforts from 19 institutions.
Our final sample consisted of interviewees from 13 doctoral or research institutions, 6 masters colleges and universities, and one associates college, based on classifications made by the Carnegie Classification of Institutions of Higher Education. The interviewees in our sample discussed implementation efforts in the following departments: • 10 physics departments • 3 biology departments • 2 math departments • 2 chemistry departments • 2 engineering departments • 2 institution-wide reform

B. Interview protocol
Semistructured telephone interviews were used to gain a detailed understanding of the implementation process from the perspective of these key informants. Each interview lasted approximately 1 hour and interviewers asked respondents to describe how they learned about SCALE-UP, how SCALE-UP affected the department and institution, and the classroom, pedagogy, and support structures. The interview protocol can be found in the Supplemental Material [39].

C. Data analysis
The interviews were audio recorded and transcribed for analysis. The project team of three lead researchers, a graduate student, and post-doctoral researcher read several interviews together in order to develop and refine a coding scheme to address the research questions. The coding scheme listed potential challenges and enablers that arose from the interviews and/or the literature. These factors were initially organized by implementation level to indicate whether the factor primarily involved an individual, a department, or the institution. We also indicated whether the factor primarily influenced the initiation stage of getting the reform started, the sustainability or spread stage of promoting continued use, or had a general influence throughout the process. In addition to using a tally system to indicate the presence of these common factors, the coding scheme also included places for written summaries to identify key ideas or nuances that were not easily counted.
During analysis, the coder marked whether each factor had an enabling, neutral, or challenging influence.
• Enablers help promote the use of SCALE-UP. For example, an administrator devoting funding to reconstruct the classrooms. • Neutral factors were actions or items that the interviewees mentioned but they did not connect to a positive or negative influence on the reform effort. For example, some interviewees mentioned collecting data but did not further discuss what was done with the data. Since these interviewees did not indicate what impact collecting data had on the implementation of SCALE-UP, we coded it as a neutral influence. • Challenges impede the initiation or continued use of the reform. For example, interviewees mention lack of funds and/or classroom space limiting the growth of SCALE-UP at the institution. Each interview was read and coded by at least two of us. Coding was compared and discrepancies were resolved.

III. RESULTS
While we found common enablers and challenges across the 21 SCALE-UP initiations, no single initiation story emerged. We note this "not one size fits all" finding as an important result of our study. The ways in which successful implementers went about their task varied significantly across the cases studied, even though it may have lead to similar final outcomes. So for the reader wondering how best to implement a radical reform, the short answer is multiple approaches can work depending on your local situation. "All reform is local." We begin this results section by describing two stories that represent two very different types of initiation stories: a bottom-up implementation and a top-down implementation. Then, we summarize the enablers and challenges that appeared most frequently across the 21 initiations we studied. Since we chose successful sites, none of the challenges these sites faced were insurmountable and we report ways interviewees overcame difficulties. Examples of how faculty dealt with challenges will be brought in as relevant. All interviewees will be referred to with male pronouns to protect anonymity.

A. Case study 1: Top-down implementation
The top-down SCALE-UP implementation at Institution R was the natural progression of incremental movement toward more active instruction. Institution R is a large, public institution with a population of approximately 52 000 students. This more selective university has very high research activity.
Our interviewee was hired as the associate dean to help the college of biology think about its teaching. He formed a curriculum task force focusing on reinventing introductory biology classes and continued to deepen his own interest in and knowledge of biology education. He mentions while these discussions were happening, he continued "to be involved in the summer institute for undergraduate biology education at the same time and so I was continually being reacquainted with the ideas of backward design, the ideas of active learning, the ideas of inclusive teaching, team based teaching, the social nature of teaching" which formed a backdrop to his thoughts and motivations.
The curriculum task force involved most biology faculty members, at least peripherally. This meant he involved a large number of people with the change, including both instructors and administrators. On this coalition he strongly advocated for SCALE-UP as a self-described "Beichner disciple." He first heard the SCALE-UP developer speak at the National Academy Summer Institute and his message resonated with his classroom experiences. After his presentation, he continued reading and hearing about the reform. When he heard him talk a second time, he was ready to take action. He recalls, "With his inspiration, it didn't take much. We were at a tipping point already and so it didn't take much for us to latch onto this as a good structure for what we wanted to do." There was a classroom management group already experimenting with studio classrooms on campus and, conveniently, they planned to build one right next to a space controlled by biology. Biology contributed their space to make a larger studio classroom and thus gained priority access. He reflects on how well this worked out, "It's all of this serendipity, you know, a little bit, but also it's a slow, incremental pushing but it's also been the willingness to be pushed." While these two groups took the initiative to begin the movement, they appealed to the university for funding. Their request was well received since they had strong advocates in the Office of the President, from faculty (especially in biology) and the knowledge of a room design expert, who helped persuade administration to fund classroom construction under the condition that it would be a general purpose space.
After making the investment, the university made it a priority to populate the rooms quickly and they developed a vision and strategy to do so. In the pilot phase, they just assigned faculty to the rooms, which helped mitigate any possible resistance. He recalls, "At the very beginning nobody was beating down the doors to get into these spaces, you know, so they just assigned people to it. And to the best of my knowledge, I only know of one single case in which a person who got assigned to the space asked to move back into a traditional classroom". Preliminary pilot research revealed that students and faculty members reacted positively to the round tables and increased engagement and felt that student learning was enhanced. They decided to continue using the classrooms while conducting quasiexperimental studies to examine how SCALE-UP contributed to students' academic engagement and learning outcomes. Since the classroom is a university-controlled space, faculty from all disciplines use it and tend to encourage their colleagues to use it, thus empowering broad-based action beyond an isolated department.
Initial pilot classes ran smoothly and the university used these short-term wins to produce more change by constructing a new building of studio classrooms. This new building could accommodate thousands of students and now the university uses their dozens of classrooms to spread publicity and promote active teaching. He noticed, "It has become one of the highlights for recruiting freshmen, period. You know, they march through that building, little tour group after tour group, they stop in the classroom." The university developed an active learning initiative that encourages interactive pedagogies in any classroom space, which he attributes to SCALE-UP; "Now the active learning classroom, physical space, I think has been an important catalyst for valuing active learning, period." He has also given tours to legislators and explains, "what we do in this kind of space, and it does, it helps the university be quite distinctive, at least so far, from other public institutions of higher education in the state." Furthermore, the university has hosted active learning conferences, which helped inspire other institutions to adopt the reform. The active classrooms appear to be changing the culture of the university, now anchored as a permanent fixture in the institution.

B. Case study 2: Bottom-up implementation
Institution U is a public institution with approximately 20 000 students. This institution has a high percentage of undergraduates and research has been increasingly important to faculty.
The key initiator at Institution U was an initially untenured physics professor who became familiar with SCALE-UP from the literature and the SCALE-UP website. The idea of building SCALE-UP classrooms was "just discussion" until chemistry hired a faculty member who had taught in a studio environment. These two faculty members started having conversations with the physics department chair and formed the guiding coalition for the reform. The department head gave permission to convert an unused lab room to a small 22-student pilot SCALE-UP section. He explained, "initially it was easy for us to take this particular lab room, because it was not really heavily used in our schedule… we just kind of took that room and stripped it back a little bit, added some projectors and there's additional whiteboards on the wall, and used that." He recalls that SCALE-UP complemented other instructional reforms happening at the university to lower the drop failure withdrawal (DFW) rates.
"This was also wrapped up for us that first year with an NSF… grant across our entire college, so all of the science departments, and it was aimed at freshmen students reducing-the DFW rates, particularly in our math courses because they were very high-and a few other initiatives, some of which dealt with the classroom, was focused a good bit on including more inquiry type activities." In the pilot implementation, they collected data including a concept test, attitude survey, and open-ended questionnaire to evaluate whether the reform was worth continuing. The numbers were too small to reveal a statistically significant difference compared to traditional courses, however, students provided positive feedback on the survey. "The students across all classes were very much saying we would like to have more of this, more collaboration and time, more time in the lab doing the labs when we're studying the materials… so it was what kind of led for us to continue." The short-term win, in the form of positive student feedback, kept things moving forward.
The next semester, our interviewee taught a second pilot section of both studio and traditional courses to allow for a control group. He separately had NSF funding to develop inquiry-based labs and the integrated lab aspect of SCALE-UP made teaching with SCALE-UP especially appealing since he could integrate this other project. While teaching SCALE-UP, he noticed improved conceptual understanding on higher order multiple-choice questions on unit exams, which caused him to buy into the model and seek more information. He shared results with the "dean at the time [who] very much did a lot of data based decision making, really looking at the numbers" and demonstrated a drop in failure rates and more students earning B's.
He recalls, "With the tie to the NSF trying to lower the DFW rate and adding the inquiry based and the things we were trying to do with that grant, the data was very nice and so at that point we began receiving more and more funding in the department to slowly change things over." Consolidating gains, the department converted more and more classrooms, mostly old lab rooms, until 100% of physics was taught in studio. He started slowly including more instructors and supported them with summer professional development. However, growing student enrollments made it was hard to deal with large class sizes, especially when some students lacked strong math skills and independent study skills. Thus, the department decided to revert algebra-based courses into a mixed mode that included some lecture. He explains, "I think the longer class periods and not meeting as frequently became overwhelming to them. I think outside of class they were becoming very frustrated because their class time was just not efficient".
To spread active learning across campus, the interviewee from Institution U now works in the Office of Institutional Effectiveness and helped his college run a program one month before school starts to train new faculty in active learning techniques. He explains, "we have our new faculty receive funding to come in across all departments within our college, a month early, so they get an additional month's salary… it focuses not only on studio classrooms but any type of active student learning technique." This community of active learning advocates originated with the "NSF project on campus that was looking for change… so we were meeting as a team, those of us involved in that grant, and just kind of sharing the best practices of what was working in our classrooms." While other departments do not use SCALE-UP in a strict sense, they have incorporated more active techniques.
SCALE-UP has become a permanent fixture in physics and administration supported it throughout with funds for renovation and equipment. Frequent changes in university administration have affected how research was valued over teaching on campus but SCALE-UP is an important part of how physics is taught at Institution U. He explains, "It's solid at this point; there will be no turning back. Again, it's just waiting on not all professors engage the model fully. The new ones absolutely do. I think it will become stronger and stronger as the older ones retire and leave the university. There was a point that I worried about that but we're well beyond that." Now that we have given detailed examples of how two institutions underwent the process of reform, we will examine the remaining data based on the most common enabling influences that emerged from our analysis.

C. Key factors surrounding reform: Enablers and related challenges
Enablers are factors the interviewees identified as helping them implement, sustain, and spread the use of SCALE-UP at their department and/or institution. During analysis, two of us individually coded each interview then differences were resolved via discussion. These differences were usually minor and were used to clarify and refine code definitions.
Initial codes were developed in an a priori list by the research team based on our experience with SCALE-UP implementations and familiarity with the literature. Additional enablers were added to the a priori list as they emerged through the analysis. Each time a new enabler was added to the list, we returned to previously analyzed interviews and updated coding as necessary.
We followed a similar procedure with the challenges. Since we studied only successful implementations, none of these challenges halted reform completely. However, we expect that all implementation efforts, both successful and unsuccessful, may experience similar difficulties to differing degrees. By studying successful efforts, we are able to identify not only the challenges but also ways that these challenges have been overcome. Many of the challenges mentioned by interviewees arise when the above enablers are not present (for example, lack of administrative support, lack of funding, lack of faculty interest), thus we will integrate discussion of both. The numbers listed referred to the number of interviewees who explicitly mentioned each factor as important, as either an enabler or challenge to overcome.
Key factors surrounding implementation that emerged from the interviews are listed in Table I. All the enabling and challenging factors uncovered by our interviews are related to these main categories and will be discussed in further detail. We will describe each key factor in the order Interaction with outside SCALE-UP user 14 Multi-member initiation team Faculty resistance 13 Culture supports active teaching Antagonistic culture 11 Enthusiastic champion 11 Professional development and faculty support Lack of faculty support of most commonly mentioned by interviewees as an important factor that facilitated reform or as a challenge that sites overcame somehow. Each key factor will include examples of how it aided or impeded reform, relevant quotes, emergent recommendations, and connections to Kotter's model. Note that just because an interviewee did not explicitly mention a factor does not mean it was not present for that implementation. We did not specifically ask about most of these influences, rather they emerged from our data and thus are only reported below if explicitly brought up by the interviewee. It can be assumed, therefore, that the actual number of implementations encountering each factor is equal or higher than what is reported.
1. Administrative support (N ¼ 20) Since SCALE-UP requires major structural changes, it was not surprising that sixteen interviews specifically discussed strong administrative support, while four mentioned securing administrative support as an initial challenge. However, as we see below, an unsupportive administrator is not an insurmountable barrier. Out of our 21 implementations, supportive department heads backed the efforts of 10 implementations and higher administrators supported 9 efforts (some efforts were supported by administrators at both levels).
In four cases, administrators actively took a leadership role in initiating the reform effort. For example, at Institution O, the department head wanted to improve undergraduate education and approached two faculty members who had visited NCSU and other active learning classrooms a few years prior. The dean provided initial funds and charged them with starting a pilot section. The faculty member from Institution O explains, "the dean at the time… was looking for a feather in his cap and looking at the initial results of the pilot program, we said, you solved the problem, so go 100% percent… he championed it, and so there was no problem getting that initial bunch of money." In four cases, administrators provided the funding and made classroom renovations possible, but were less involved with execution of the reform. Three administrators had a past interest in innovative teaching methods and used their expertise for detailed planning and/or teaching with the reform. Four administrators were under pressure to improve instruction and gave faculty members permission to try the reform. Two administrators actively publicized the reform to help recruit other faculty members to use it.
We frequently found administrative support connected with concerns around retention, accreditation, or other measures of success. Leveraging these concerns and the ways in which SCALE-UP addresses them can get administrative support. An instructor from Institution C elaborates, "The more we can get in and out in four years, the higher degree of funding we get from the state. So there's a big push to improve that and so there's a lot of effort from the provost's office to look at what that would take, and that includes improving the teaching methods, you know, the pedagogy in those first year courses, the larger lecture based courses, increasing student engagement, freshman seminars, all kinds of different things." He elaborates that accreditation helped too, "our regional accreditation, we went through a big general education review, assessment and all those things, there's some external institutional accountability issues that are helping, too." Since administrators at Institution C were thinking about these issues, the initiation effort had administrative buy-in.
Related challenges and solutions.-Four interviewees mentioned having challenges associated with administrative support, either gaining it initially or protecting the reform against a changing administration. One faculty member got the attention of administrators by encouraging fellow faculty members to assist in calling for change. Another creative instructor grabbed an administrator's attention by posting an invitation on the faculty list-serv for anyone to visit her class. When the provost visited and saw how the classroom structure complicated the active learning process, he immediately wanted to take action. He remembers, "he came running down to the front of the room and said, 'What do you need? What can I do? This is great but this room is ridiculous, what do you need?' And so he put together a team to help design what the room should be." However, sometimes the challenge is more substantial. Even in cases where multiple enablers were present, lack of departmental support can halt forward progress. A physics instructor at Institution U successfully implemented SCALE-UP in his department, had collected data to document its value, and had planned to further expand SCALE-UP use. Progress stopped when her university named a new dean. He explains, "at the end because that could have had an impact on my future within the department. I had to be careful on that. I was not in a position to go forward and without him for supporting it… I mean not all faculty in the department were excited to go here." Another instructor successfully implemented SCALE-UP at his first institution but then pushed for SCALE-UP at a new institution and ran into a barrier when his department head did not want studio classrooms. It was not until he became department head that he was able to modify existing laboratory rooms into SCALE-UP spaces: "It wasn't until I was chair that I could make it happen so that seemed to be like the pivotal thing. It didn't really require much or any money to say, let's take existing lab space… It was having a chair willing to say this is going to happen." Recommendations.-Kotter [23,24] lists "creating a sense of urgency" as the first step in the change process. In our study, it was important to convince people that change is important. To gain the support of faculty and administrators, it is imperative to find out which arguments resonate with people and argue accordingly to convince people that change in worthwhile. For administrators, this may involve trying to reduce DFW rates, distinguish their university with an attractive classroom that symbolizes a switch to more modern instruction methods, or an effort to increase retention to appease accrediting agencies or get increased funding from state legislators.
Also, support can come in multiple ways and from different directions. If a department head is unsupportive, try to gain support of upper administration or vice versa. To guard against issues with changing administrators, take a precautionary approach where you plan in advance to include multiple administrators on your guiding coalition. With multiple administrators involved, one or two people changing positions will have less of an impact.

Document and leverage evidence of local success (N ¼ 19)
Eighteen interviewees mention demonstrating positive outcomes of implementation with quantitative and/or qualitative data as helpful. Six mentioned the negative outcomes of student complaints as a difficulty, with only one mentioning it as a barrier that needed to be overcome. Quantitative data included course or exam grades, DFW rates, attendance records, and/or scores from established disciplinary concept inventories (for example, the Force Concept Inventory [40] in physics). Qualitative data included student evaluations, informal anecdotes from students or instructors, and/or a close examination of student work. When trying to convert science faculty at higher education institutions, keeping data has been mentioned as a way to soften resistance, sustain, and grow reformed teaching efforts [41][42][43][44].
Promising to document the reform effort was a common way to help secure administrative permission to start a pilot section. Usually fellow scientists appreciate objective analysis, so proposing a SCALE-UP pilot as an educational experiment may help convince administrators to provide permission and initial funds. For example, one instructor from Institution B says he got permission "to do the chemistry majors course so that we could collect data. And I think that that data… softened people's resistance to it." Some instructors have found that keeping data was incredibly important in proactively battling potential pushback, especially if the administration changes. For example, one institution has been using SCALE-UP in the majority of their introductory physics courses for fifteen years but changes in the administration put the program under threat. He believed data had been the most important piece in keeping the program alive; "This is where the assessment data really helps us out." While data can be leveraged to get more funds and/or space and help defend the reform against changing administration, they are not always sufficient to ensure survival of a SCALE-UP implementation. The instructor from Institution F describes this concern. He mentions that data are, "how we've been defending things and that's how we've been going forward. Hopefully, that continues to be an effective method of defending it. But if somebody's not willing to look at the data and it doesn't, that's not going to help." In addition to systematically collected data, many institutions found anecdotal feedback can be influential, especially in situations where class sizes are too small to establish statistical significance. For example, in two departments, SCALE-UP use was significantly expanded because of student requests, including the example of Institution U provided in the case study above. At these institutions, students mention enjoying the interaction on their course evaluations. Some faculty found teaching more "fun" in this mode and enjoyed establishing relationships with students. The interviewee from Institution C summarizes; "[Instructors] definitely report back that their classrooms seem to be going better; the students are more interested, they're more engaged; not all of them are doing scholarship of teaching and learning research… but trying to teach better so they seem happy with the direction that their changes have gone in. So yes, anecdotally, the teachers are excited." Related challenges and possible solutions.-While student resistance was never strong enough to dramatically slow or halt the implementations in our study, six interviewees mention that their students grumbled about learning in this unfamiliar fashion. For example, the instructor at Institution T reports, "It's clear that students are frustrated, some students are frustrated by being asked to take more responsibility for their own learning so you get that kind of stereotypical response where 'they're not teaching me,' right, because they're not lecturing to you." Meaningful instructional change is often accompanied with complaints and resistance from students. While these reactions are not unexpected, there are still proactive steps you can take to reduce the impact of negative responses. For instructors, it may help to explicitly discuss how and why classes will be different when they are taught in a new environment. A faculty member from Institution R recalls, "we realized that part of what our job needed to be was being very open with students about why we were doing what we were doing, what the research was, and then acknowledging that they would be uncomfortable." Based on a review of research on student resistance to active learning, Seidel and Tanner [45] concluded resistance is often less a reaction to the pedagogy than to negative instructor behaviors in the classroom, such as sarcasm, tardiness, and unresponsiveness to students. Students may pick up on and exaggerate a faculty member's own resistance to interactive instruction. Thus, it is important that instructors believe in change and feel comfortable and supported using new techniques. Professional development (as discussed in factor 8) can help.
Fortunately, interviewees reported that student resistance goes down over time as students get used to the new instructional style or as it becomes widespread within the department or at the institution. Thus, as mentioned earlier, converting all classes to an interactive style can help minimize student resistance because usually people taught with traditional methods are most resistant. "Juniors and seniors, were the ones who struggled the most because they were used to, they had it down, how to do a lecture… and they kind of resented the switch, I think. It's a lot easier now that incoming freshmen have classes in the room, you know, before they move up." Having past students work as teaching assistants can also help mitigate resistance [46]. As one instructor from Institution R explains, "Having the reality of somebody who's been through this class before and has figured out how to learn well in that modality and has gone through the frustration… we found that early on to be very helpful in helping students come to grips with having to do something in class rather than just sit there." This problem might also resolve itself naturally as instruction gets modernized. Interviewees also mentioned that they have noticed that resistance has naturally gone down in the past 15 years since today's students are more familiar with technology and more likely to have been exposed to active instruction in high school. According to Instructor 2 from Institution D, "We messed with their vision of what the college experience was supposed to be… that was much, much truer back in 2000 or so and I think now I guess many more high school classes are very active and engaging and the students are kind of more used to it…. in the last year… I don't notice that same kind of resistance at all." Recommendations.-Universities are increasingly held accountable for quantitative measures of success. Collecting data and evidence of local success are important throughout the reform process, following Kotter's 6 [th] step of "generate short-term wins." Proposing to start SCALE-UP as an educational experiment can help get administrative approval and collecting data before the reform serve as a baseline comparison. Data can be leveraged to get additional funding or build more classrooms. Data can also be used to promote sustained use of the reform when the administration or support structures change.
Although collecting systematic data is important, it is not always sufficient to create successful change. People who are resistant to change frequently ignore, or find reasons to discount, arguments based on data. In our study, we found that anecdotes, including stories about student success, can be particularly influential. Anecdotal stories can appeal to a broader range of interests and the important emotional side of decision making. In addition to anecdotes, qualitative data may be helpful for classes where the numbers are too small to lead to statistically significant differences in quantitative data.
Negative feedback from students can be hard to hear but interviewees found several ways to mitigate this resistance. For example, using past students as teaching assistants, setting expectations on the first day, and introducing these methods early in a student's educational career.

Funding (N ¼ 17)
Fourteen interviewees mentioned significant institutional funding to support implementation including modifying the classroom environment, staffing costs, and instructional resources, while three secured funding from outside sources. Institutional funding came from a variety of places. Examples of "pots" of institutional funds used in SCALE-UP implementation included facilities upkeep money, furniture funds (Institution F), retention and recruitment funds (Institution C), departmental-level funding from program fees (Institution B) and funds allocated by state legislatures. The representative from Institution R explains, "Like any other building, this was funded in a partnership with the legislature." A department head from Institution B secured funding by submitting a proposal in a competitive process designed to allocate program fees. "The provost instituted program fees and we were asked to submit a proposal…. Since I had been thinking about this stuff, we [the biology department] had a very mature proposal, and we get all of the program fees." Other interviewees used minimal funding to run a pilot to collect data to be leveraged for additional funding. The interviewee from Institution Q explains, "I had a relatively small class and we just used a regular classroom and we got a small grant from the university to use some equipment and some grad student time and try it out. And we did. We gave a presentation of our results and the provost really liked it and she put a lot of money into it and, all of a sudden, we were off and running." In addition to the initial investment for classroom renovation, it is important to consider ongoing expenses. Two institutions prepared cost estimates to prove that ongoing staffing costs would be equivalent to a traditional model. For example, at Institution T, "they wanted to make sure was that it didn't involve any [additional] staffing. Staffing is the largest expense at our school. And as long as the staffing was adequate, they were basically happy." One way to do this is to integrate the laboratory into the studio setting, which can help maintain the same number of faculty contact hours. An instructor from Institution F explains, "by raising the lab cap and lowering the classroom cap, as far as they were concerned it was the same number of instructor hours that we were paying for, so they went along with it." As with any reform, SCALE-UP often requires extended monetary support, although these expenses may be equivalent or less than funding required for a traditional model. Thus, ongoing institutional funding is often more sustainable and predictable than one-time grant funding. In our study, institutional funding was used for reoccurring costs like staffing and room upkeep. Funding can also provide incentives for faculty to invest time changing their teaching. For example, faculty might feel more supported in trying something new if they are provided with release time, teaching assistants, and/or instructional technology support. Furthermore, since administrators are usually in charge of distributing internal funding, institutional funding almost always involves securing internal buy-in.
Thus, securing internal funding from a department head and/or university-wide administrators is helpful and can increase the chance of long-term success. To do this, it is important to communicate the vision and justify these experiences using arguments that would appeal to administrators. For example, a representative from Institution R justifies the expense by explaining how their room can be used for student meetings and ad hoc study space when not scheduled for classes. "That's important for people when they're considering investing in this kind of space. That it is used all the time for all kinds of things… not just for teaching and learning. So it really saves you money in the long run." Administrators may also feel their investment is worthwhile if they view the space as something that distinguishes their university as progressive and special, which can attract other sources of funding down the road. Four interviewees discuss how they use the classroom to advertise reformed teaching and distinguish their university. One institution explains how they built excitement around the opening of the new classroom, "When it first opened, we did a ribbon cutting, we invited people… the alumni foundation to a deans' council, you know, to a teaching demonstration of SCALE UP." They continue to use the space for faculty meetings to spark interest. University E volunteered their classrooms to serve as a "poster child for good teaching at their university" by allowing visitors, VIPs, and alumni to visit the classrooms and attend classes. This resulted in additional funding from alumni, more instructors interested in teaching with these techniques, and prestige for the university. They found many successful outcomes from their "conscious effort… to attract more money, more people, more converts to the movement." In conclusion, institutional funding is important because our interviews found that external funding, through a research grant or national funds, was quite rare (in only three of our 21 cases) and even then it never fully funded the reform. In addition to the case from Institution E, one source of external funding came in the form of a dissemination grant by the SCALE-UP developer that provided some initial funds but had to be supplemented with university funds to buy furniture and keep the reform going. The third case of external funding was a complementary grant to develop curriculum for inquiry-based labs that could be used in SCALE-UP.
Related challenges and solutions.-Instructional reform and classroom renovations require money that institutions and/or administrators may not have or may not be willing to spend. Five interviewees were concerned about securing funding for the initial classroom, a future classroom, or a better classroom. This is especially a problem if administrators think there are cheaper alternatives to increasing student engagement. An instructor from Institution B explains, "I would like to get more support from the university, but they see it as costing money… their solutions for dealing with retention was to create online courses." While it is common for administrators to have this concern, plan staffing and room renovation so the reform can be financial feasible. While financial constraints may make it hard to modify the classrooms, there are ways to find funding and make the most of existing funds.
Often, converting classrooms that can fit typical university introductory classes (100 þ students) is very expensive. Thus, even supportive institutions may only have active learning classrooms suitable for smaller class sizes. Interviewees mentioned not having active learning space on campus to fit the number of students they typically teach. A particularly striking example was a biology instructor at Institution C who spearheaded the effort to get classrooms renovated but the introductory classes he teaches were too big for him to ever use the reformed space. He explains, "the classroom only holds about a hundred and my classes are typically two hundred and so that's a personnel issue and if they start giving me class sizes of only a hundred, who's going to teach the other hundred students, you know?".
Other interviewees ran into issues because they only had permission and funds to build SCALE-UP classrooms in renovated spaces that were strange shapes (e.g., L shaped) or too cramped, and thus not ideal. Another instructor was able to renovate a classroom hidden away in a basement so it did not gain interest of his department. The interviewee from Institution G recalls, "There wasn't much of a reaction within the department because… no one or not that many people were using it… There's a lecture hall upstairs and so the people that were already comfortable with the lecture hall continued using that rather than going into the basement." Thus the shape, location, and visibility of the classroom also matter.
Recommendations.-A radical reform requiring structural changes necessitates some funding. External funding was rare, never covered the entire cost, and was always temporary. However, it can be used to leverage internal funding and vice versa. Obtaining internal rather than external funding appears related to the long-term success of the reform because it is accompanied by administrative buy-in and can cover reoccurring costs, such as staffing, training, and technological support.
One way to make classroom renovation feasible is to prioritize which classroom elements are most critical to reform success and minimize the expensive extras. For example, many institutions have found that round tables and whiteboards are the most important aspects of promoting collaboration. Thus, they found studio-style classrooms could be funded with existing facilities upkeep funds and still be pedagogically effective. At institution L, faculty demanded some classrooms that, conveniently, are less expensive to build; "A couple of departments have taken some of their own money and renovated some rooms, which I would call [a light version], in that they have movable tables and chairs with additional screens but they don't have all the [expensive features]." Many times renovating a lab room or rearranging a preexisting classroom can lead to basic benefits. An instructor at Institution G recalls that his department head was able to donate some funds then "it was just a matter of getting cheap enough computers to sort of fit within the budget. Drilling the tables down ourselves, putting holes in them rather than having people do it." Although inadequate classrooms challenge successful SCALE-UP use, these classrooms can serve as a transitional space and/or pilot rooms for collecting data to convince administrators to invest more money in a higher quality space. To get started, compromise as much as you need to in order to get a pilot classroom. Then, you can look for opportunities to upgrade, for example, search for classrooms that need renovating, the availability of new funding sources, other innovative teaching efforts on campus to collaborate with, etc. For example, Institution J was able to build adequate space when an old classroom was damaged in a flood and needed renovation anyway. In this case individuals were ready to seize the opportunity afforded by the flood and advocate for the rebuilding of a new type of classroom.
In the meantime, there may be ways to maximize the current situation. For example, if class sizes are too large for an available active-learning classroom, it is possible to adopt a mixed model where students spend time in a traditional lecture classroom and some time in a SCALE-UP classroom. While not ideal, Institution B reports "the fact that we didn't insist on having all these large courses taught fully in the active learning classroom means that we can get a lot of students in there a week. At least once a week, they're taking advantage of it." To conclude, forming a budget and finding funding should be discussed alongside Kotter's third step of "form strategic vision & initiatives." Figuring out funding is often linked to other issues, including the location and appearance of the classroom, and providing professional development or faculty support. The goals of the reform affect the distribution of funds and the prioritization of classroom elements. All of this should be discussed and articulated.

Interaction with outside SCALE-UP user (N ¼ 15)
Fifteen interviewees mentioned the importance of interactions with the reform developer and/or other secondary users. Talking to experienced users and/or conducting site visits to established implementations was valuable to (i) gain information about the reform, (ii) use credibility from other sites to convince administrators to try the reform, and (iii) get advice during the reform effort.
To gain information about reform, five initiation efforts had representatives visit established SCALE-UP-style classrooms (including North Carolina State University, University of Minnesota, and Rensselaer Polytechnic Institute) to see successful studio instruction in action before deciding to initiate their own efforts.
Nine interviewees brought the reform developer or an outside user to campus, either in person or virtually. For example, Institution T had two established secondary users come to campus to get advice about the room and hired the same architect who worked on the project at the University of Minnesota. Institution J "beamed in" the reform developer for a videoconference with faculty and administrators and found this "really catapulted the faculty members' interest beyond what we were saying, because he was able to answer the questions about what worked, what didn't work." Another person noticed this "to be able to say this person from this big regional institution helped us… gives a certain amount of credibility to what we're doing." Interpersonal interactions allowed SCALE-UP implementers to get personalized advice on logistical matters. In addition to the SCALE-UP developer, some consulted educational departments or other users on their campus (N ¼ 2), while another reached out to other geographically accessible users (N ¼ 1). For example, Institution U worked with another university in their state to share advice and information. This became a mutually beneficial arrangement: they helped start the reform at the other institution and also received advice as the implementation progressed. Their visits to campus "started an opportunity for us to talk back and forth and share how those hurdles, struggles as we were going through… of what's working for you and what's not working." Recommendations.-Having the developer of the reform on campus to forge personal connections, provide details, and field questions is likely to have a positive impact. In other research [21,47], personal connections were found to be one of the strongest influences on faculty in their decision making around educational innovation. Building these connections is likely worthwhile. The SCALE-UP wiki [48] contains a directory of current reform users, often with contact information. This is a good place for people interested in SCALE-UP to view existing resources and start connecting with other sites.
For the disseminators of reform, the lesson here is that investing the time to relate directly and personally with potential secondary implementations-and not just relying on written materials to disseminate ideas-is wise. The importance of interpersonal communications is also supported by diffusion of innovations research: the credibility of and the level of contact with the change agent are both positively related to the decision to adopt [49].

Multimember initiation team (N ¼ 14)
None of the successful reforms studied was initiated by a single individual acting alone. We believe this to be a highly significant finding. Fourteen interviewees explicitly mentioned the helpfulness of a multimember team with their reform effort. The team approach versus the individual approach provides numerous advantages. For example, instructors often do not have control over room design and thus, in most cases, administrative support is required for the creation of a SCALE-UP room. An instructor from Institution C claims both faculty and administrative interest were both critical to the success of the reform effort, "there was a convergence of a certain handful of faculty leaders making noise and the administration actually understand-ing… You need to have someone in that position who gets it because otherwise you're just constantly explaining it to somebody who doesn't quite get it and doesn't buy in." In three cases, the multimember team preceded the existence of SCALE-UP. In one case, this took the form of a departmental curriculum reform committee, and in two cases, this took the form of a university planning committee. This worked especially well since there was a pre-established structure in place to talk about education, brainstorm ideas, and share experiences. For example, an instructor from Institution U explains that a previous NSF grant paved the way for SCALE-UP because a group interested in innovative teaching was already "meeting as a team, those of us involved in that grant, and just kind of sharing the best practices of what was working in our classrooms. And it just slowly comes more from that of what was successful." Another institution revived a building committee that had been planning a new facility with writable walls and flexible seating back in 2005 but those plans were halted when the recession hit. When the economy improved, "our facilities director, since we worked on the building in the university center, decided that we were still going to take the ideas that was going to be in that active learning classroom and build out one of the rooms… to look like that. So that came online in 2011." When looking to assemble a reform team or guiding coalition, Kotter [23,24] recommends you include people with power, expertise, credibility, and leadership. Many of our interviewees gave examples of teams that included members in these categories. The interviewee from Institution C explains, "the strategy was to put together a team of people that were kind of leaders in this kind of learning on campus; there are little isolated pockets of us in different disciplines trying to do something besides lecture. So those people were asked to be on the committee to help design it." Instructor 1 at Institution D gives another example of a successful committee started by the dean which included "representatives from physics, mathematics, chemistry and also our staff, our technical staff, because they were the ones who would have to of course do a lot of the footwork on this, not the you know the pedagogical work, but they would have to do the nuts and bolts of putting these laboratories together; so it was quite a collaborative affair." Institution L describes their "work team that included a registrar and facilities and the associate provost and myself and the academic technology person." Related challenges and possible solutions.-Since we have discussed the lack of administrative support in factor 1, a potential remaining challenge is gathering faculty support, which affected four implementations. In one case, vocal faculty opposition limited the number of classrooms built in the new building on campus, a decision they are now "stuck" with. Instructor 1 at Institution E reported faculty resistance when their department head suddenly changed all the introductory math courses to a SCALE-UP format at once, however, resistance did not impede reform. More commonly, two institutions report that they have faculty that do not fully "buy in" or embrace the model, but it did not create a major impediment.
To combat faculty resistance, first, it is important that faculty know about SCALE-UP implementation efforts. It is helpful to put newly modified rooms in central, visible locations and expose people to them at every opportunity. An instructor from Institution C discusses how they used the faculty bulletin to advertise demonstrations and workshops in SCALE-UP classrooms. He says, "as soon as that came out, like the announcement and the pictures, and stuff, people got interested, like 'I want to teach in there, I want to check out this place because the resources are much nicer than some of the older rooms'." As mentioned above, another strategy is just assigning instructors to the classroom so they have the opportunity to try it. Providing an incentive to the first "pioneers," in terms of a stipend and/or faculty release time can be particularly effective. People can also confront reluctant faculty with the supporting data and let the evidence speak for itself.
It also is critically important to change the culture of the department to make sure instructors are supported to teach in the SCALE-UP environment. This includes providing training and making sure the workload is equivalent, if not easier, than teaching in a traditional course. For example, make the reformed pedagogy the standard format for common curricula, assessment, and homework assignments. Identify faculty interests and concerns then enable action by removing barriers. Refer to "culture supports active teaching" and "professional development and faculty support" below, factors 6 and 8, respectively.
Recommendation.-In 14 of 21 successful implementations studied, interviewees mentioned that strong collaborative teams (rather than individuals) helped initiate reform. Reform advocates are best advised to start by building a team or leveraging an existing team. Successful multimember teams usually include both administrators and faculty members. This is helpful because administrators have control over funding, scheduling, and room design to execute the reform while the instructors will be the ones using the classrooms and they have a better idea of the practical realities of implementation with students.
Kotter's second step [23,24] recommends creating a guiding coalition with people that have power, expertise, credibility, and leadership. In a higher education context, Wieman et al. [31] specifically found that involving a supportive, respected chair or head with authority as well as senior and junior faculty helped facilitate change. That finding is echoed in our study. Repeatedly, interviewees reported administrative approval and/or funding was required to modify the space. This confirms Kotter's discussion of needing "position power" on the guiding coalition. Efforts went especially smoothly when a guiding coalition had members with expertise in educational reform, classroom design, and/or teaching in a studio environment. More details about the importance of administrative support and an enthusiastic faculty champion are discussed in enablers 1 and 7, respectively.

Culture supports active teaching (N ¼ 13)
To insure the continued use of SCALE-UP, twelve interviewees mention ways that they have consciously changed the departmental or university culture to value active learning, for example, in hiring and/or promotion. Often this begins with hiring people with knowledge and/or experience in innovative teaching. "We did make a concerted effort about ten years ago to start hiring in chemical education and most of our new faculty… have had their PhDs in chemistry with emphasis on chemical education." Institution T uses the presence of studio classrooms to hook the interest of instructors who value interactive teaching. "People come here because they want to teach, they want to form relationships with students. So that's where I think active learning absolutely aligns with our mission… to build relationships and that it's through those relationships that students learn." Having the whole department convert can help facilitate this process and develop a community around active learning. When everyone teaches this way, they understand the new mode better. A faculty member from Institution F reflects, "I think we haven't encountered much [faculty] resistance up until this point… is because other people have started teaching in the rooms and they also saw the efficacy of having students be able to interact with each other and be able to engage the students a little bit more actively in the classroom." This can lead to empathy amongst instructors and building a community around reform. As the interviewee from Institution T observes, "when the whole department converts, then everybody supports each other, people understand how much work it is, they value that in terms of tenure and promotion stuff and merit things, so all of that." A department at Institution R discusses how this is part of peer evaluation for retention and promotion. When instructors evaluate each other, they ask questions about instructional materials, instructional setup, and want to make sure that instructors are doing more in the classroom than lecturing for 50 minutes. He comments, "It's become so common now that those are things that are actually looked for in instructors in our department." Having all the introductory classes taught uniformly can also lower student resistance. As instructor B notices, "student resistance is higher when you've got one faculty trying to do something and no one else is, and when it's a more broad scale and it's been going on for a longer period of time, because students generally, once they experience active learning, they'll often come to prefer it because they recognize the value in it." Someone from institution K said they only teach physics with studio so "our students don't know any better, they just accept that's the way physics is done." Finally, our study revealed that it is important to ensure that structural changes (including the reformed classroom) preserve this reformed culture. Our interviewees noticed that when some instructors teach in reformed spaces, even temporarily, it causes a permanent transformation in their instructional methods. Two instructors noted that when they teach higher level courses in traditional spaces they were more conscious about reducing lecture time and fitting in more clicker questions, white boarding, and promoting student-student interaction. However, in the case of SCALE-UP, the reformed classroom space could change classroom behavior and can even impact teaching practices in traditional classrooms. For example, a faculty member from Institution T noticed, "It just has a fantastic ripple effect because most people who teach in a space like that and change one class end up changing other classes even if they're not in that same kind of space." Someone from Institution R adds, "I think it's helped us embrace the idea of active learning to the point that, it's almost like the room itself catalyzed the pedagogical changes". The interviewee from Institution R reflects, "This is one of those cases where 'if you build it, they will come' is really true… much of our behavior is driven by the physical environment… so the room itself can be a partner or a barrier in whatever it is that you want to do, and for active learning, it's an enormously powerful partner." These observations stand in stark contrast to many reform movements where once the training ends or faculty lose interest, there is no way to guarantee that reformed teaching will continue.
The redesigned classroom may need to be accompanied by other structural changes, for example, in scheduling (for longer class sessions) and budgeting (i.e., ongoing funding for teaching or learning assistants, professional development opportunities). Eventually, if these structures are in place, SCALE-UP style instruction can become the status quo, unlikely to disappear even if key figures leave. An instructor from Institution F comments, "this is something like the entire department is committed to, so like we've all, we're all part of it, I mean it's just part of how we schedule, it's just how we do things." Related challenges and possible solutions.-While our successful sites all had cultures relatively amenable to instructional innovation, two interviewees had to wait for department heads to move on before the culture allowed for change. The interviewee from Institution U discusses how one administrator can make a difference in compensating for a department that focuses on research productivity. "They admit to us very boldly: I was hired for my research skills and no one cared about my teaching experience during the interview. I am here to be research productive… But teaching is still important to the dean that we currently have. And she makes it clear to these new hires that, yes, you were hired for your research but I personally have high teaching expectations." Changing a department or university culture is not a trivial undertaking. However, most of the other factors we previously identified can help: start by getting administrative and faculty support, which might involve collecting data, then proceed from there. Even if change cannot happen right away, take steps to pave the way for future change by assembling a guiding coalition so when something changes, the reform effort is ready to take off.
Recommendations.-Echoing Kotter's seventh (sustain acceleration) and eighth steps (institutionalize change), it is important to change the surrounding university culture so the reform becomes a natural part of how courses are taught. The more the campus community knows about the change, the easier it will be to gain support for the reform and help it eventually become part of the campus culture. Place reformed classrooms in central locations and use them as tools to inform faculty, administrators, legislators, and prospective students about the reform. Holding faculty orientations and/or professional development courses in the space may be a great way to spark interest and alert the campus of the reform movement. Accompanying structural changes in scheduling and budgeting also reflect the values of maintaining studio style instruction as the status quo.
In addition to leveraging the physical classroom, we recommend that departments consider use of active learning techniques in the hiring and promotion process. Change becomes permanent and institutionalized when it is anchored in the local culture and norms are shifted to value innovative teaching. Converting all introductory courses to a studio format can accelerate building a department that engages in active learning. If the majority of faculty are involved in the studio effort, they can exchange advice and resources to grow a reform community. Eventually, many of the implementations we studied reached a point where SCALE-UP was "just how they did things" in that department and thus would be unlikely to disappear even if key people moved away.

Enthusiastic champion (N ¼ 11)
In eleven efforts, the perseverance of a reform "champion" helped advance the change despite logistical and other obstacles. In his work on diffusion of innovations theory, Rogers [50] defines "champions" as individuals within an organization who stand behind an innovation and break through opposition. Usually these are educated, charismatic, well-respected, and well-connected people within the organization. Many of the SCALE-UP implementations we studied did not happen overnight and required creativity to overcome obstacles. These champions had enough patience and persistence to persevere with their goals even when administrators were not listening and/or funding was not available.
An instructor at Institution Q was able to rally instructors when administrators repeatedly denied his request for a SCALE-UP space, claiming there was no room for a classroom since the classrooms were at capacity and no other instructors were interested. Then he started talking to instructors, showing them the video of the University of Minnesota's Active Learning Classrooms and SCALE-UP classroom designs from NC State. He recalls, "When somebody tells me 'no,' I practice guerrilla warfare. I'll get grassroots support so then we can go back and say we have a number of instructors that are interested in this and we got it done." Indeed, getting other instructors interested proved to be successful in getting the attention of an administrator capable of making the reform happen.
In a similar effort to gather grassroots support, an enthusiastic champion from Institution C benefitted from being knowledgeable (since he does biology education research) and well connected: "Because I do a lot of faculty development, I was able to have a forum by which to diffuse those ideas, and get those out and to seek out others who were doing it on campus… these people who are doing these things quietly by themselves, let's pull together and see if we can be a voice that has a little more clout." This is the instructor mentioned earlier who used a university list-serv to gain the attention of his Provost to invite him into his classroom. Once he was interested, he had the connections and expertise to form an effective guiding coalition and put the reform in action.
Instructor 1 from Institution D was so passionate about starting studio classrooms that he invested time and even bought lab materials with his own money to start a pilot. It did not take long before his determination and enthusiasm sparked the interest of those around him and he quickly got more support for his efforts. He and another faculty member "just did everything ourselves, did not bother the staff, and staff became so intrigued by what we were doing that they volunteered to prep for us and to help us develop the labs." It also led to fellow faculty becoming interested in teaching in the room.
In one case, it took four years and a lot of persistence to get funding for studio classrooms. Someone from Institution T tried to get funding from university legislature for two years but "when that failed, then we went to the president and the provost and said we're going to need to do this internally… they ended up providing the funding once they were able." While he does not expect to get such a large amount of funds again, he says smaller classrooms take fewer resources and realizes "that's just a question of them queuing up and when we have the resources or facilities, doing them." Without someone persistent and enthusiastic enough to pursue multiple funding sources for multiple years, this reform effort could have easily shut down.
Recommendations.-While "lasting change is not created by lone visionaries" (Ref. [34], p. 75), individuals can plant a seed that flourishes with the right nourishment from colleagues and institutional leaders. While involving administrators and others is quite critical for success, many of the situations documented here started with one or a few instructors who were committed to improving their practice. Having enthusiastic and creative individuals willing to preserve can help overcome challenges securing administrative support and funding. Further, we note that reform efforts were often supported by groundwork laid by previous efforts, even when earlier efforts were less impactful. This implies that even small efforts to encourage active learning can plant the seeds for later efforts to draw on. While having an enthusiastic champion can help overcome many hurdles, recall that a team effort was another important enabler. None of the implementations we studied were driven by a single champion: the enthusiasm of a single individual is likely not sufficient for this level of change.

Professional development and faculty support (N ¼ 11)
Offering professional development to accompany the reform helps ensure that instructors are supported in best using the redesigned room. Teaching in SCALE-UP requires many skills that are different than those required in lecture-based courses. As the interviewee from Institution F explains, "You really have to be comfortable with a lot of interaction with the students, you have to be comfortable with kind of giving up control of your class-room… some people really have struggled with that." Thus it is important to make sure instructors enter the classroom prepared. As an interviewee from Institution C noted, students notice when instructors are not using the room appropriately. "There are varying degrees of authentic implementation of some model of using SCALE UP for active learning… and the students were pretty astute in noting that. And so if it was just to do something the old way but with a little more sparkle, you know, like with the technology… they notice that." Instructors should feel supported in making instructional changes. A department head at Institution B discusses the kind of support he provides: "Nobody tells the faculty to go out and do this, we provide resources… we have [an] instructional designer you can work with, we can provide innovative TAs to make that happen, and we have our teaching specialist. Everybody's busy, and if it just comes down to the faculty member's expected to just magically come up with time to do this, it's not going to happen." Professional development opportunities can help instructors to incorporate active learning and embrace the SCALE-UP model. Institution T instituted training proactively; "we wanted to know why they needed that space. It's a huge capital investment so we wanted to know that whoever was teaching in there was converting their courses appropriately." At Institution M, certain department heads assigned faculty members to attend a mandatory training program. The interviewee from Institution M recalls that this initially caused faculty resistance, recalling professors who came into a faculty workshop with the attitude of, "we've been punished by making us do this [but] interestingly, those two people… ended up being the biggest converts by the end of it, and one of them has continued to serve on the advisory committee for the project." He noticed that once these skeptical faculty members converted, it significantly lowered the rest of the department's reticence since faculty "outside the chemistry education club" tried it and had a good experience.
Four institutions compensated at least the first initiators with either release time or a stipend. Institution T explains, "we had a competitive grant program for people to apply to teach in the space and if they received a slot to teach in the space, we gave them course development stipends over the summer to do the course development work." Providing faculty with incentives or financial compensation to participate in professional development can help, especially when starting the movement. This competitive and prestigious process effectively lowered faculty resistance and built a critical mass to ensure the continued use of reform. As a representative from Institution N explains, "The fact that people were being paid to participate in this training I think lowered a lot of people's resistance. And then that got the numbers way up, as far as who was going to participate… Also, participation in the faculty development program is based on an application. It's a competitive process so there's a certain level of commitment just in applying for it." When administrators offer compensation, it also shows faculty members that their efforts to improve teaching were recognized and appreciated.
Other departments and/or university faculty professional development centers host optional workshops before or during the semester. One institution pays new faculty a stipend to attend a month-long new faculty orientation that includes workshops on interactive pedagogies to expose them to the possibility and benefits of teaching using SCALE-UP. A department head from Institution B decided to have a session on SCALE-UP at the faculty retreat since all introductory biology classes were taught in that mode. The department head elaborates, "We had breakout groups and we worked on some of these issues that we still haven't resolved entirely, but it was a great way to get faculty buy-in. It's not topdown; we're trying to get through the idea that we're scientists, and evidence is very important for us in our research, and why not take that same approach with our teaching." Another option is to provide common course materials and/or weekly coordination meetings. An instructor from Institution O noticed brand new instructors appreciated having an initial structure that taught them how to teach. "What happens with the SCALE-UP as we run it, with coordination meetings, with activities manual, it gives those new faculty members a head start on getting into learning how to teach." Instructor 1 from Institution D noticed how SCALE-UP promoted collaboration between instructors, not just the students in the class.
"Anybody new coming in really has to get information from people who have taught the course and it's become really a collaborative of people sharing new ways that they've tried to get across a concept or new ways of approaching a laboratory or how to integrate the laboratory with the concepts… it becomes a lot of fun." Some departments (N ¼ 2) have a formal SCALE-UP course manager who ensures the sections have common curricula including exams, homework assignments, and class slides. This reduces course preparation time so teaching with SCALE-UP is equivalent (or easier) than teaching a traditional course. [34] A final option is to provide informal faculty shadowing and mentoring. For example, an instructor from Institution H explains, "We don't exactly have a formal professional development but… they could sort of be a SCALE UP TA, kind of like a TA in the semester before; then they could help out in the classroom and they could see how it goes and they would get familiar with the material." Other institutions pass along written materials; "Any time we have a new faculty member come in now, I still take to the print shop that original training that I used just for physics faculty for that week in the summer and it gives them all of the worksheets and things that I developed in-house." Related challenges and possible solutions.-Two interviewees mentioned faculty who felt they were unsupported. At Institution K, the faculty member felt it took more time to develop a research-based, innovative curriculum to accompany the reform. While this may be true for the first iteration of reform, usually once a curriculum is developed, the teaching load can be comparable to teaching a traditional class. Sharing course materials and consulting outside resources (like the SCALE-UP wiki [48]) can reduce pressure on instructors.
Institution U felt that the load on instructors was especially high since their university had no graduate students so they had to rely on undergraduate teaching assistants. While they viewed this as a challenge, many SCALE-UP secondary users have found undergraduates to be sufficient. Three of the sites we interviewed trained their undergraduates based on Colorado's learning assistant model [51].
Recommendations.-Based on reports from our interviewees, having a formal professional development program in place to help instructors teach using reformed pedagogies in the reformed room has a positive impact. Active learning requires different instructional skills than those used in lecture environments. Past studies have shown that without changing the pedagogy to match the reformed room, learning gains may not improve [52,53]. We recommend that all reform efforts include some kind of professional development and support. It can range from a selective mandatory training program (where we found the prestige of the program helps mitigate faculty resistance) to weekly coordination meetings with common curricula to informal mentoring and resource sharing.
Furthermore, professional development is linked to Kotter's fifth step of empowering broad-based action. To make reformed teaching appealing to instructors, provide support structures to alleviate or eliminate some of their concerns. In the instructional reform literature, one of the most consistently mentioned biggest barriers to reform is lack of time [34]. Successful implementations benefit from materials and training to make the instructor transition to SCALE-UP less time intensive.

IV. CONCLUSIONS
As mentioned in the introduction, most of current reform literature documents the best case scenario of reform at the development site, which often benefits from local expertise, funding, release time, etc. [14][15][16][17]. This study fills the current void in the research literature by examining how secondary sites navigate the change process successfully, including what challenges they face and how these are overcome. This work demonstrates that with patience and persistence sites can radically reform the way they teach, often by taking advantage of certain enablers. Identifying these common factors give potential PER users some direction for where they can focus their efforts for initiating change and overcoming obstacles. It is one of the first studies in PER to examine the secondary implementations within a coherent framework, answering a call made by researchers who proposed theoretical change processes and wanted them utilized in practical situations [25]. It also has implications for reform developers who want their reforms to be used successfully, because it reveals the type of support or advice secondary sites may need.
While the 21 people we interviewed all had unique stories, several common factors emerged as aiding initiation, spread, and sustainability of the SCALE-UP reform. Many of our recommendations echo those made by Kotter's 8-step model. This work provides readers with examples of what those steps might look like in a higher education context. Table II summarizes how the enablers fit with the three main phases of change described by Kotter. To establish a climate for change, it was helpful to give faculty and administrators a reason to change, and to create a multimember guiding coalition that includes administrators with power to make structural changes and create visibility for the efforts. To engage and enable the organization, sites benefitted from enlisting the aid of experts, recruiting and training faculty members, and collecting data from pilot efforts that could be leveraged to spread the reform. To implement and sustain the change, it was helpful to change the culture to value active learning with strategic hiring and a continued expansion of classrooms. In order to overcome potential challenges, we found it was helpful for sites to align their efforts with administrators' concerns, mitigate student resistance, and use creative funding sources. All of these messages are critically important for secondary sites that may be intimidated by change initiatives or discouraged by past failures.
Other major findings are summarized below.
(1) There is no one right way to initiate reform. This leads directly into the most important conclusion from our study. All change is local and depends on the institutional and departmental circumstances, culture, and setting. Thus, there is no one right way to implement a particular reform. The reform effort changes depending on institutional decision making structures, availability of staff, class sizes, and local expertise. An instructor from Institution S comments, "I was familiar with SCALE UP, we have done our own variation of this… all instruction is local. You have to find what is going to work for you." We identified common enabling influences (supportive administration, background in active learning, information exchange with other users, and more) but also found different routes can still lead to a successful implementation.
We saw bottom-up and top-down implementations can both succeed as long as there is some faculty and administrative collaboration. This is more important than just having an enthusiastic champion who is willing to get external funding. While external funding can be helpful in the beginning, institutional commitment is necessary for sustainability. Thus, it is critical to put together a guiding coalition with administrators, preferably more than one, to guard against potential administrative changes. It is also important to consult faculty members so they are interested and invested in teaching in the spaces.
Thus, there is no one right way to initiate a substantial reform like SCALE-UP. Much of it depends on the institutional and departmental context, the past experiences and beliefs of relevant actions and the resources available at the university.  (2) Many challenges can be overcome Since we studied successful SCALE-UP implementations, none of these sites encountered insurmountable challenges. Common challenges fell into two main categories: structural challenges (scheduling, limited space) and cultural challenges (faculty and student resistance). Most institutions in this study found ways to mitigate cultural challenges but found structural challenges to be harder to eliminate and these sometimes limited SCALE-UP spread.
Often sites found ways to mitigate student and faculty resistance. Many faculty members used undergraduate learning assistants to help with student resistance or found that it naturally diminished over time as the instructional method became more accepted. While some interviewees mentioned uninterested colleagues, no faculty resistance was significant enough to derail the implementation.
Structural concerns with space and timing tend to be more significant challenges that could limit the spread of change within a department or institution. Large class sizes and scheduling concerns often involved factors outside an individual faculty member's immediate control and sometimes depended on higher administration. Data could help prove that the reform effort was worthy enough to preserve despite inconveniences and could be leveraged to gain additional funds or space.
(3) Reform takes time It is important to remember meaningful changes do not happen quickly [44]. Many institutions began change efforts by spreading information about active learning pedagogies and starting conversation. They then had to wait to implement SCALE-UP until the right administrator took over, a source of funding became available or classroom space became available for renovation. Even if all the elements are not present at once, it is important for reformers to still plant the seeds by connecting with others in the organization. In all of the cases we studied, there were events that took place well before the implementation of SCALE-UP that enabled initiation. Frequently, the stories we heard indicated that fuel was being developed for some time before a final spark finally ignited the reform effort. Efforts, which at the time may appear to be fruitless, may later turn out to be a critical foundation for a successful implementation later on.

V. IN CLOSING
One of the reasons we selected SCALE-UP for this study is because it is a radical instructional reform that has successfully spread into multiple disciplines at educational institutions around the world. We believe some of the reform's success is due to the significant structural modifications that accompany the change, with a visibly modified classroom environment and sometimes an altered schedule.
While there is no "magic formula" for initiating structure-changing reforms, gaining awareness of common enablers and suggestions for overcoming common challenges may increase the chances of successful secondary implementations.