Use of research-based instructional strategies in introductory physics : Where do faculty leave the innovation-decision process ?

During the fall of 2008 a web survey, designed to collect information about pedagogical knowledge and practices, was completed by a representative sample of 722 physics faculty across the United States (50.3% response rate). This paper presents partial results to describe how 20 potential predictor variables correlate with faculty knowledge about and use of research-based instructional strategies (RBIS). The innovation-decision process was conceived of in terms of four stages: knowledge versus no knowledge, trial versus no trial, continuation versus discontinuation, and high versus low use. The largest losses occur at the continuation stage, with approximately 1=3 of faculty discontinuing use of all RBIS after trying one or more of these strategies. Nine of the predictor variables were statistically significant for at least one of these stages when controlling for other variables. Knowledge and/or use of RBIS are significantly correlated with reading teaching-related journals, attending talks and workshops related to teaching, attending the physics and astronomy new faculty workshop, having an interest in using more RBIS, being female, being satisfied with meeting instructional goals, and having a permanent, full-time position. The types of variables that are significant at each stage vary substantially. These results suggest that common dissemination strategies are good at creating knowledge about RBIS and motivation to try a RBIS, but more work is needed to support faculty during implementation and continued use of RBIS. Also, contrary to common assumptions, faculty age, institutional type, and percentage of job related to teaching were not found to be barriers to knowledge or use at any stage. High research productivity and large class sizes were not found to be barriers to use of at least some RBIS.


I. INTRODUCTION
Recent decades have seen large expenditures of time and money on research and development related to the improvement of introductory, college-level science, technology, engineering, and mathematics (STEM) courses.Significant empirical research has shown that student learning can be substantially improved when instructors move from traditional, transmission-style instruction to more student-centered, interactive instruction [1,2].In physics, much of the research and development efforts, until very recently, have been dominated by small groups of curriculum developers who research and develop their own curricular products [3].Thus, there exist a relatively large number of named curricula-see Table I and the bibliography of Ref. [38]-that have been empirically shown to improve student learning in many of the problem areas identified above.Examples include Peer Instruction [19,39], Interactive Lecture Demonstrations [11], Tutorials in Introductory Physics [32], Cooperative Group Problem Solving [7,8,40], and Workshop Physics [36,37].The developers of these curricula most commonly disseminate their work through talks, workshops, and publications.For example, Mazur, developer of Peer Instruction, noted that between 1996 and 2009 he gave over 300 talks about Peer Instruction and that 18 700 copies of his book about Peer Instruction [19] had been shipped-including 12 700 free copies [41].This represents approximately one free copy for each of the roughly 13 000 physics faculty employed in all four-year and two-year colleges in the United States [42,43].

II. PURPOSE AND RESEARCH QUESTIONS
Although there has been substantial effort and money involved in the creation, testing, and dissemination of these high quality research-based instructional strategies (RBIS), little empirical work has been done to understand the extent to which faculty have been engaged in learning about or implementing these strategies.This paper uses the results of a Web survey of physics faculty in the U.S. to answer the following questions: (1) what percentage of faculty exit at each stage of the innovation-decision process, and (2) what characteristics of faculty and their teaching situations correlate with exiting or remaining in the innovation-decision process?Answering these questions will help curriculum developers and other change agents understand the impact of their past efforts and work towards improved dissemination efforts.It will also help to identify barriers to the use of RBIS that may need to be addressed in order for RBIS to become widely used.

III. BACKGROUND AND THEORETICAL FRAMEWORK A. Innovation-decision process
Much research and most dissemination strategies suggest that faculty use of RBIS is based on individual decision making [44][45][46][47][48]. Rogers proposes that this sort of decision-making occurs over time in a series of five stages [49]: knowledge about the innovation, persuasion about the benefits of the innovation, decision to use the innovation, implementation of the innovation, and confirmation of continued implementation of the innovation.According to Rogers, the types of information and support required by a potential adopter vary at different points in the innovationdecision process.Similarly, the important characteristics of the potential adopter and their concerns about how well the innovation will fit with their situation might vary at different points in the innovation-decision process.For example, a decision to seek knowledge about a RBIS may arise due to an instructor's belief that their current instructional practices could be improved.The implementation decision may be based on their perception that the RBIS would work with the typical class size at their institution.The decision to discontinue use after trying a RBIS may be based on their perception that continued use of the innovation will reduce research productivity.

B. Research-based instructional strategies in physics
This study focuses on 24 RBIS that have been developed for the teaching of introductory quantitative physics at the college or university level.These strategies are listed in Table I along with references where interested readers can turn for additional information.In this section we describe some of the common features of these strategies and briefly summarize the research base supporting the efficacy of these strategies.
The set of 24 RBIS was meant to include all of the RBISs that were widely available for faculty use at the time the survey was administered (fall of 2008).The list was developed by us and vetted in several ways, both with experts in the field of physics education research as well as in pilot testing of the survey.We are aware of only one RBIS-the PhET simulations [50] from the University of Colorado-that was inadvertently excluded.
Several authors have compared the features of RBISs for introductory, college-level physics and have concluded that the commonality is that all strategies promote some form of active engagement [38,51].Redish identifies four characteristics of an active engagement course ( [38], p. 118): (1) the course is student centered-it is focused on what the students are actually doing in class; (2) laboratories involve guided discovery where students observe phenomena and build their ideas; (3) the course may include explicit training of reasoning; and (4) students are expected to be intellectually active during the class.According to Redish, these active engagement elements may be incorporated into the entire class or into particular parts, such as the recitation or laboratory sessions.These RBIS has been tested in a variety of ways by individual researchers.Perhaps the most robust finding from this body of research is that student conceptual understanding of core physics topics is significantly and consistently higher in courses using active engagement methods compared to courses using traditional, lecture-based methods [51].
As an example of how active engagement can be implemented in an introductory physics course, consider the

Research-based instructional strategy
Active Learning Problem Sheets [4,5] Activity-Based Physics Tutorials [6] Context-Rich Problems [7] Cooperative Group Problem Solving [8,9] Experiment Problems [10] Interactive Lecture Demonstrations [11] Investigative Science Learning Environment [5,12] Just-In-Time Teaching [13] Modeling Physics [14,15] Open Source Physics [16] Open Source Tutorials [17] Overview, Case Study Physics [18] Peer Instruction [19,20] Physlets [21,22] Ranking Tasks [23] Real Time Physics/Tools for Scientific Thinking Labs [24] Scale-Up, Studio Physics [25][26][27] Socratic Dialog Inducing labs [28] Thinking Problems [29] TIPERS [23,30,31] Tutorials in Introductory Physics [32] Video-Based Labs [33,34] Workbook for Introductory Physics [35] Workshop Physics [36,37] RBIS of Peer Instruction [19,20,39,52,53].Peer Instruction was developed by Mazur for use in his large lecture introductory physics courses at Harvard University.In a Peer Instruction class, the instructor delivers short lectures (7-10 minutes) followed by a multiple-choice conceptual question.Students first think about the question and answer it individually (often using a personal response, or clicker, system), then discuss their answer with a nearby classmate, and, finally, revise their answer.Based on student responses to the multiple-choice question, the instructor may decide to move on to the next topic or to continue with the current topic.Mazur summarizes the results of the Peer Instruction instructional method as follows ( [53], p. 51): ''Data obtained in my class and in classes of colleagues worldwide, in a wide range of academic settings and a wide range of disciplines, show that learning gains nearly triple . . .Most important, students not only perform better on a variety of conceptual assessments, but also improve their traditional problem-solving skills.Also, data show that such interactive engagement helps to reduce the gender gap that exists in introductory physics classrooms.''

C. Diffusion of research-based instructional strategies
Very few studies have investigated the level of knowledge about or use of nontraditional teaching practices by faculty who teach introductory college-level STEM courses.One of the primary sources of information in physics comes from a Web survey [54,55].Results indicate that almost all faculty (87%) say they are familiar with one or more RBIS and approximately half of faculty (48%) say that they currently use at least one RBIS [55].Faculty report lack of time as the biggest impediment to using more RBISs [54].
Similar information about levels of knowledge about or use of educational innovations are available from Web surveys in geosciences and engineering.In a survey of geoscience faculty, MacDonald, Manduca, Mogk, and Tewksbury found that fewer than 25% of faculty report regularly using interactive techniques (other than lecture with questions or demonstrations) [56].They also found that faculty who teach large introductory courses use interactive techniques much less frequently than faculty who teach smaller courses.In a survey of engineering department chairs about faculty knowledge and use of student active pedagogies, Borrego, Froyd, and Hall found that 82% of faculty report having knowledge of student active pedagogies and 47% report adopting such pedagogies [57].They found few significant differences by institution type.
Walczyk and Ramsey conducted a survey of science and math faculty in the state of Louisiana about their use of learner-centered instruction [58].They found that traditional instructional practices were the dominant instructional styles in science and math classrooms at both teaching and research-oriented institutions.They also found that faculty who teach larger classes were less likely to use learner-centered instruction.Participation in Louisiana Collaborative for the Excellence in the Preparation of Teachers professional development activities was very weakly correlated with use of learnercentered instruction.
The results of the surveys described above provide some estimates about the level of knowledge and use of RBIS by college STEM faculty.They do not, however, provide much information about why instructors are making these choices.Henderson and Dancy propose that an instructor's selection of an instructional style results from the interaction of characteristics associated with the instructor (individual characteristics) and characteristics associated with their instructional context (situational characteristics) [59].Thus, to understand faculty practice it is important to consider multiple individual and situational characteristics.There is currently research-based evidence available about only a small number of possible characteristics.These characteristics are summarized in the following items.
(a) Attending professional development workshops.Many RBIS are disseminated via workshops.Some evidence suggests that faculty who attend these workshops are more likely to know about and use RBIS [58,[60][61][62].It is, of course, difficult to experimentally measure the impact of such workshops since there is often significant self-selection among faculty attendees.(b) Focus on research over teaching.The reward structure in higher education that tends to favor research over teaching is often cited as a barrier to the use of RBIS [2,48,[63][64][65].If this is the case, one might expect faculty at teaching-oriented institutions to use more RBIS than faculty at research-oriented institutions.However, several of the studies mentioned earlier did not find differences in teaching styles between different types of institutions [56][57][58].(c) Class size.Large class sizes have consistently been found to be barriers to innovative teaching [48,56,58,59].(d) Gender.Female faculty tend to use more methods consistent with interactive engagement, such as utilizing extensive student discussion, and generally have less transmissionist-oriented teaching philosophy than their male counterparts [66][67][68][69].(e) Age.Older faculty are often less innovative teachers than younger faculty [68,70].Although many of these characteristics have been considered separately, we are not aware of any previous studies that have sought to use a comprehensive set of situational and personal characteristics to determine the relative strength of correlation with knowledge and use of RBIS.We are also not aware of any previous studies that have done this for several stages of the innovation-decision process.

A. Web survey and sampling procedures
A Web-based survey was developed and administered to a national sample of physics faculty from three different types of institutions (four-year colleges that offer a graduate degree in physics, four-year colleges that offer a bachelor's degree in physics as their most advanced physics degree, and two-year colleges).Institutions within each of the three types were randomly selected.Once selected, all eligible faculty were identified from the institution Web site and/or through contact with the department chair and invited to take the survey.Only one graduate-degreeoffering institution was dropped from the study because insufficient information was available online and the department chair did not respond to Emails or phone calls.Faculty were eligible for the survey if they had taught an introductory quantitative course in the past two years and were full-time or permanent employees (i.e., faculty who were part-time, temporary employees were not eligible for the survey).By quantitative physics we are referring to the algebra-or calculus-based introductory physics classes that often go by the names of ''college physics'' or ''university physics.''These have been the target courses for most RBIS and also represent the largest physics enrollments.
The Web survey consisted of 61 questions, including several questions designed to identify personal characteristics of the respondent and their perception of their situational characteristics.Respondents were also asked to rate their level of knowledge about and use of 24 specific RBISs that have been developed and disseminated for use with introductory quantitative physics (see Table I).On the survey, each RBIS was identified by the RBIS name and the name(s) of the developer(s), but no description of the RBIS was provided.The response rate of 50.3% resulted in 722 usable responses, 701 of which were used in this analysis (21 are not included because they did not respond to any knowledge or use questions).The survey and sampling procedures are described in more detail elsewhere [55].

B. Measuring explanatory variables
The Web survey was designed with questions to identify potentially important variables related to faculty and their teaching situation.Based on the survey items, 20 potential explanatory variables were developed (see Table II).

C. Connecting survey responses to innovation-decision stages
In the survey, for each of the RBISs, faculty were asked to select one of the following choices: (1) I currently use all or part of it, (2) I have used all or part of it in the past, (3) I am familiar with it, but have never used it, (4) I've heard the name, but do not know much else about it, and (5) I have never heard of it.Respondents who answered ''1'' were categorized as users of the RBIS.Respondents who answered ''2'' were categorized as former users of the RBIS.Respondents who answered ''1,'' ''2,'' or ''3'' were categorized as having knowledge of the RBIS.Respondents who answered ''4'' or ''5'' were categorized as having no knowledge of the RBIS.
These self-reported levels of knowledge and use were then related to Rogers's stages of the innovation-decision process as shown in Table III.Since we did not collect any information about respondents' perceptions of the innovations, we combined Rogers's persuasion, decision, and implementation stages into a single stage that simply indicates whether a respondent had tried a RBIS.Also, although not a feature of Rogers's framework, our preliminary analysis of these data suggested that there may be important differences between respondents who used several RBIS and those who used few RBIS [71].This is a potentially important distinction since there is research evidence to suggest that student learning outcomes can be improved through thoughtful combinations of several RBISs [72,73].Thus, we created the ''high user'' and ''low user'' categories.Since we have no theoretical or empirical basis upon which to draw the dividing line between high and low, we sought to split the groups as evenly as possible.Thus, high users are defined as those who use 3 or more RBISs (above average use) and low users are defined as those who use 1 or 2 RBISs (below average use).

D. Identifying significantly correlated variables at each innovation-decision stage
At each stage in the innovation-decision process, faculty were divided into two groups.For example, at the first stage, there is a group of faculty who are aware of one or more RBISs and there is a group of faculty who are not aware of any RBISs.We wanted to determine which of the variables were differently represented between each pair of groups.Relationships among explanatory variables and group membership were tested by a Pearson's chi-square test.
Screening of individual variables to identify potential predictors (variables that are not evenly distributed between groups) should not be confused with multiple testing [74].Here we are testing whether there is a relationship of the membership variable with group membership.This is not the same as testing if there is any difference between the groups in any of the 20 variables, which is equivalent to testing hypotheses about at least one difference, and requires adjusting the significance level for individual comparisons to avoid inflation of the experimental (global) type one error rate.In our case, we are testing the association of each variable with the membership category to make conclusions only about the importance of that particular variable.Thus, modifications of significance levels for individual comparisons are not necessary and it is appropriate to use the 0.05 statistical significance level as a cutoff.

E. Importance of significantly correlated variables
In addition to identifying which variables are correlated with group membership at each stage of the innovationdecision process, we are also interested in the direction and importance of each of the variables.That is, which variables vary the most with group membership and what condition of the variable is related to high knowledge and/or use?Since many study variables are correlated with one another, understanding the importance of individual variables requires development of a statistical model that can control for other study variables.In determining appropriate statistical techniques, it is important to note that the faculty groups and most of the variables in the study are categorical and need to be analyzed using methodology appropriate for their type.Therefore, commonly used techniques such as principal component analysis, discriminant analysis, or factor analysis cannot be used here, as they are applicable only for explanatory variables normally distributed at each level of group membership.Logistic regression is commonly advocated as preferable to these other techniques when dealing with categorical data [75][76][77].As a rule of thumb, the overall sample size necessary to protect a logistic regression analysis from inflated probabilities of a type one error (i.e., finding that a particular variable is significant when it is not) should be at least 5 times larger than the number of cells in the contingency tables [78].In our study, the 11 significant variables, each with two categories after dichotomization, would require a sample size of at least 110.Given that our sample size of 722 is much higher than this rule of thumb minimum, we can safely conclude that the sample size is sufficient for a logistic regression analysis.Within logistic regression, odds ratios are commonly used to describe the importance of study variables.Odds are simply the proportion of respondents in a particular group with one condition of a variable compared to the proportion of respondents in that group with another condition of the variable.For example, we can calculate the female:male odds for the knowledge group by taking the number of females divided by the number of males.This will tell us the odds that a randomly chosen member of the knowledge group would be female.Odds, though, depend quite a bit on the distribution of states of the membership variable in the sample of survey respondents.Continuing with the example, in this survey there were many more male respondents than female respondents (reflecting the overrepresentation of males in the population of physics faculty).Thus, the female:male odds in the knowledge group is not particularly meaningful on its own.In this case we have chosen to compare the odds from one of the membership groups to the odds of the other group at the same stage of the innovation-decision process.Thus, in our example, the odds ratio would be the ratio of female: male odds for faculty in the knowledge group to the female:male odds for faculty in the no-knowledge group.An odds ratio equal to 1 means that there is no difference in the female:male odds between the groups.An odds ratio larger than 1 means that the female:male ratio in the knowledge group is higher than in the no-knowledge group.That is, females are disproportionately more likely to be in the knowledge group.Similarly, an odds ratio less than 1 means the female:male ratio in the knowledge group is lower than in the no-knowledge group.
Odds ratios and 95% confidence intervals were determined from a logistic regression model containing only the variable in question.The odds ratios calculated in this manner are identical to ratios that could be calculated using the raw data, except that the confidence interval is helpful in identifying the range of possible ratios.For example, a confidence interval that contains an odds ratio of 1 suggests that the variable is not significantly different between the two groups.
These odds ratios using a single-variable model are useful for some applications, but one difficulty interpreting the results is that many of the variables are correlated with other study variables.For example, the gender variable is correlated with several other variables, such as working at an institution with a physics Ph.D. (females are less likely) and attending the New Faculty Workshop (females are more likely).We wanted to control for these other variables to better understand the relationship of gender to group membership.
Thus, for each stage in the innovation-decision process we developed a nominal logistic regression model that retained all of the significant variables.From this model, an additional set of odds ratios and 95% confidence intervals were calculated for each variable.

A. Faculty progress in the innovation-decision process
Figure 1 shows the distribution of faculty at each stage of the innovation-decision process.As can be seen from the figure, 12% of faculty report not having knowledge of any of the RBISs surveyed.That is, they leave the innovationdecision process prior to the knowledge stage.Another 16% of faculty have knowledge, but have not tried any RBIS.The largest to exit, 23% of faculty, discontinue use after trying.Thus, of the faculty who have tried a RBIS,

B. Relationship between explanatory variables and group membership
Table IV shows which variables are significantly correlated to group membership at each stage of the innovationdecision process.As can be seen from the table, 10 of the variables are not significantly correlated with group membership in at least one stage of the innovation-decision process.These nonsignificant variables are (see Table II for an explanation of the acronyms used) CRSE, ENC, DGRE, GOAL, JOB, PEER, RANK, RSH1, RSH3, and YEAR.Once controlling for other study variables, INST was also found to be nonsignificant.This leaves us with nine variables that are significantly correlated with group membership in at least one stage: READ, ATND, NFW, MORE, GEN, SATF, PSTN, RSH2, and SIZE.Only READ is significant at more than two of the four stages.This is consistent with our expectation that faculty are likely to have different needs and concerns at different stages in the innovation-decision process.

C. Importance of significantly correlated variables
model using only the single variable as well as regression model that retained all 10 of the significant variables (READ, ATND, NFW, MORE, GEN, SATF, PSTN, RSH2, SIZE, INST).This multivariable model controls for all of the other study variables.Variables with 95% confidence intervals that do not overlap with the number 1 are those variables that are considered significantly different between the two membership groups.
From these figures we can see the importance of the variables and how they change at each stage of the innovationdecision process.For example, consider the NFW variable.This variable identifies whether or not an instructor has attended the Physics and Astronomy New Faculty Workshop.This workshop has been run 1 or 2 times annually since 1996 and each year attracts between 25% and 50% of all new physics faculty at four-year colleges and FIG. 2. Odds ratios and 95% confidence intervals for knowledge versus no-knowledge groups.The 95% confidence intervals for the NFW variable extend off the chart.A confidence interval that includes ''1'' suggests that the groups are not significantly different.FIG. 3. Odds ratios and 95% confidence intervals for tried versus not-tried groups.The 95% confidence intervals for the NFW variable extend off the chart.A confidence interval that includes ''1'' suggests that the groups are not significantly different.universities in the U.S. Faculty typically attend in their 2nd or 3rd year of their first tenure track appointment.Looking at odds ratios between the knowledge versus no-knowledge groups (Fig. 2), we see that, after controlling for other variables, the NFW:no-NFW odds are over 10 times greater for the knowledge group than the no-knowledge group.The NFW odds ratio for tried versus no-tried groups is almost 7 (Fig. 3).These are both quite large odds ratios.For the continued versus not-continued group, the NFW odds ratio is only about 1.5 and the confidence interval suggests that the odds ratio is not statistically different from 1 (Fig. 4).Thus, we conclude that the large difference in the NFWodds that we found for the knowledge and tried stages is no longer present at the continued stage.This nonsignificant NFW odds ratio (not statistically different from 1) is repeated for the high versus low user stage (Fig. 5).These results suggest that the New Faculty Workshop is successful in helping faculty develop knowledge about new FIG.4. Odds ratios and 95% confidence intervals for continued versus not-continued groups.A confidence interval that includes ''1'' suggests that the groups are not significantly different.FIG. 5. Odds ratios and 95% confidence intervals for high versus low user groups.A confidence interval that includes ''1'' suggests that the groups are not significantly different.
instructional strategies and motivating faculty to try these strategies (or, at least, maintaining the high motivation level that faculty have coming into the workshop).This result is consistent with prior work [60].These results also suggest, however, that the workshop does not help faculty continue use of these strategies or combine more than two strategies to become high users.Faculty who attend the workshop are just as likely as nonattendees to discontinue use of new instructional strategies and to be low users.

VI. LIMITATIONS
The results and conclusions reached here are only as meaningful as the explanatory variables developed.One of the limitations we encountered in developing the levels for the explanatory variables is that in some cases it was impossible to develop levels that had respondents reasonably distributed between them.This was a significant problem for the GOAL variable.For GOAL, 84.2% of respondents rated both problem solving and conceptual understanding as very important.Since almost all of the respondents are in a single level, it is not surprising that this variable is not useful in predicting group membership.
A similar situation occurred with the variable ENC.Here, 65% of respondents rated the department as very encouraging and another 27% of respondents rated the department as somewhat encouraging of efforts to improve instruction.Only 8% rated the department as either neutral or not encouraging.Given that most higher education institutions are not known for their support of efforts to improve instruction, we suspect that this 92% level of selfreported departmental encouragement reflects the inability of the survey to adequately capture this construct.
Given the poor distribution of respondents within these two variables it would not be appropriate from this study to conclude that instructional goals (GOAL) or departmental encouragement (ENC) are not correlated with knowledge about and use of RBIS.It is possible that better measures of these variables would identify such correlations.Both of these variables have been identified in other studies as related to innovative teaching [48,[79][80][81].
The other significant limitation of this study is that all of the survey data are self-reported.We suspect that the data here overreport faculty knowledge and use of RBIS.For example, some faculty reported being users of a particular RBIS on the survey, but at the same time described instructional practices that were not consistent with that RBIS [55].Unfortunately, we do not have a good way to estimate the impact this might have on the results presented here.

VII. DISCUSSION
A. What percentage of faculty exit at each stage of the innovation-decision process?
The percentage of faculty exiting at each stage of the innovation-decision process is shown in Fig. 1: 12% have not developed knowledge about any RBIS, another 16% have never tried any RBIS, and another 23% have discontinue use after trying.
An important way for change agents to look at these numbers is to identify where the biggest losses occur.In raw percentages, the biggest losses are faculty who have tried at least one RBIS but no longer use any RBIS.Another way to look at these numbers is to consider the percentage of faculty who have made it to a given stage but do not continue to the next stage.From this perspective, approximately 1=8 of faculty (12%) have not developed awareness of any RBIS.Of the faculty with knowledge, approximately 1=6 of faculty (16%=88% ¼ 18%) have not tried any RBIS.Of the faculty who have tried a RBIS, approximately 1=3 of faculty (23%=72% ¼ 32%) discontinue use.Thus, current change strategies seem to do a reasonably good job of helping faculty develop knowledge and motivation to try these new instructional strategies.But additional work is needed to understand and address the 1=3 of faculty who discontinue use after trying.It may be more fruitful to focus on those who discontinue use than to focus even more effort on encouraging the remaining holdouts to a RBIS.

B. What characteristics of faculty and their teaching situations correlate with exiting or remaining on the innovation-decision process?
Table IV shows the variables that are significantly correlated at each stage of the innovation-decision process.
At the knowledge stage, there are seven significant variables (READ, ATND, NFW, MORE, SATF, PSTN, INST), five of which remain significant in the full model (READ, ATND, NFW, SATF, PSTN).Notice that of these five, three (READ, ATND, NFW) are direct mechanisms for gaining knowledge.Thus, it is not surprising that faculty who have knowledge have engaged in these activities.It is notable that by far the largest odds ratios between faculty with knowledge versus faculty without knowledge are related to attending the New Faculty Workshop (NFW).The NFW:no-NFW odds are over 10 times greater for the knowledge group than for the no-knowledge group.This very high odds ratio represents the strong correlation between attending the New Faculty Workshop and having knowledge of RBIS.While only 13% of our faculty sample had attended the NFW, nearly all of these (99%) were in the knowledge group.
At the trial stage, 5 of the 7 variables from the knowledge stage continue to be significant (READ, ATND, NFW, MORE, INST) individually.Only three of these are significant in the full model (READ, NFW, MORE).While NFW and READ are mechanisms for gaining information, MORE is a measure of self-reported interest in using more innovative teaching strategies.Again, having attended the NFW continues to be the largest odds ratio, although it has decreased somewhat such that the odds are only about 7 times greater for attendees.
At the confirmation stage, there are 5 significant variables (READ, ATND, NFW, MORE, GEN), 2 of which remain significant in the full model (MORE, GEN).Notice here that variables related to gaining knowledge are no longer correlated.This suggests that, while attending talks and workshops (including the New Faculty Workshop) and reading about innovations can help faculty develop knowledge and motivate them to try innovative teaching methods, these mechanisms do not support maintained use of these innovations.Instead it is variables related to personal characteristics of interest in using more innovative teaching methods and gender that are significant at this stage.Here, the largest odds ratio is gender.The female:male odds are over twice as large for the continue group than for the discontinue group.
At the level of use stage, there are 7 significant variables (READ, ATND, GEN, SATF, RSH2, SIZE, INST), 6 of which remain in the full model (READ, ATND, GEN, SATF, RSH2, SIZE).Compared to low users, high users are more likely to read journals about teaching, attend talks and workshops related to teaching, be female, be satisfied with meeting instructional goals, publish fewer research articles, and teach smaller classes.ATND has the largest odds ratio.The attend:not-attend odds are about 4 times greater for the higher user group than the lower user group.The most likely explanation for this is that faculty who are higher users are continuing to attend talks and workshops related to teaching while many faculty who are lower users may not be actively seeking new information about instruction, but rather rely on instructional strategies that they are already familiar with.This stage is the only place that two commonly mentioned barriers to the use of innovative instruction arise: research productivity (in the form of publications) and class size.It is important to note that while these variables are significantly correlated at this stage, faculty in the low use group (who are more likely to publish more research articles and teach large classes) are still RBIS users.

VIII. CONCLUSIONS AND IMPLICATIONS
There are several important conclusions and implications from this study.In looking at where faculty leave the innovation-decision process, we see that the largest losses occur when faculty who have tried a RBIS later decide to discontinue use.This suggests that current professional development is effective at helping faculty learn about and become motivated to try innovative instructional strategies, but that it is important to find ways to reduce the amount of discontinuance.Finally, this study calls into question previous expectations about some barriers to the use of RBIS.

A. Developing knowledge and motivation to try
As shown in Fig. 1, most faculty (88%) are aware of at least one RBIS and many (72%) have tried at least one RBIS.This suggests that current dissemination practices, commonly focused on informing faculty about RBISs and convincing faculty to try RBISs, are generally effective.Indeed, when we look at variables that are significantly correlated with group membership at the knowledge and trial stages, we see that many of them relate to the common dissemination methods of giving talks and workshops (ATND, NFW) and publishing articles about new instructional strategies (READ).We also see that permanent, fulltime faculty are most likely to have knowledge about one or more RBIS.Presumably this is because their job security allows them the freedom and incentive to investigate RBIS.

B. Problem of discontinuation
Although current change efforts appear to be reasonably successful at helping faculty develop knowledge about RBISs and providing motivation to try them, approximately 1=3 of faculty who have tried a RBIS no longer use any RBIS.This high level of discontinuation suggests that more attention needs to be given to developing ways to support faculty to be successful in their implementations.
There are many reasons why faculty may discontinue use of a RBIS.One is that when learning about a RBIS (via talks, workshops, or reading), they were presented with an overly rosy picture of how well the innovation would work.Then, when they actually try to implement the RBIS they are faced with difficulties, such as student complaints [79,82,83], an inability to cover the amount of content that they feel is appropriate [59,79,84], or weaker than promised student outcomes [73,85].Another related reason is that when faculty decide to implement a RBIS they usually do not follow or even necessarily learn about all of the details of innovation use described by the developer.Instead, they invent or reinvent these details for themselves [83,86].Thus, traditional dissemination (talks, workshops, publications) should be careful to articulate potential problems, reasonable expectations, and essential features of RBIS use.This is not commonly done.
Another way to think about this problem of discontinuance is in terms of support provided during implementation.While it may be possible to foresee implementation difficulties and provide faculty with additional advice before they begin to use a RBIS, it is almost certain that additional support during initial use will lead to more successful use.One important finding from a literature review on change strategies in higher education is that successful strategies provide support during implementation in the form of performance evaluation and feedback [87].In current dissemination strategies, support and feedback during implementation are quite rare.
Finally, one factor that emerged from this work that distinguishes faculty who continue from faculty who discontinue use of innovative instructional strategies is being female.This result is consistent with the findings of others [66][67][68][69].For example, Statham et al. found gender differences in both the beliefs as well as the practices of faculty [66].Through a combination of interviews and classroom observations, and after controlling for other variables related to gender, they concluded that female faculty are more likely than male faculty to view students as important contributors to the class, use student-centered instructional strategies, and obtain higher levels of student participation.
These findings suggest that institutions should continue their efforts to promote diversity by hiring more female physics faculty.Currently (2006) only 13% of physics faculty are female and 43% of physics departments have no female faculty [88], so there is certainly room for improvement.However, we suspect that an important reason that being female is a significant predictor of continued and high use is that it reflects a set of beliefs about teaching and learning and the role of the teacher that are more commonly found in women than in men.These beliefs could help females persevere through the inevitable implementation difficulties.Since these beliefs were not captured in the survey used in this study, the GEN variable may simply serve as a (weak) proxy for a set of RBISfriendly beliefs.We advocate more work to identify these beliefs, document their distribution among male and female faculty, and identify ways to strengthen these beliefs in male and female faculty.

C. Barriers to innovative instruction
In this study, we identified faculty levels of 20 potential predictor variables.We are not aware of any previous study that has measured such a range of variables and related them to faculty use of innovative teaching methods.By doing this we are able to develop a logistical regression model at each stage of the innovation-decision process to control for potentially significant variables.We have been able to confirm some previously identified correlations between variables and knowledge or use of innovative instructional strategies (e.g., that attending professional development helps to develop knowledge and motivation).Our results, however, also call into question some previously identified correlations.Some variables are often thought to be barriers to innovative teaching but were not found to be barriers to at least low levels of RBIS use.These include age (YEAR, RANK), percentage of job responsibilities related to teaching (JOB), type of institution (INST), class size (SIZE), research productivity (RSH1, RSH2, RSH3), departmental encouragement related to teaching (ENC), and discussions with peers related to teaching (PEER).Thus, it should not be assumed that more-senior, research productive faculty, or those who teach large classes cannot or will not use RBIS.
Age (YEAR, RANK).More-senior faculty are often thought to be less innovative than younger faculty [68,70].Here, we have two proxies for age: years of teaching experience (YEAR) and academic rank (RANK).Neither of these variables were significant in the single or multiplevariable models at any stage in the innovation-decision process.Thus, we find no support for the idea that age is related to knowledge about and use of RBIS.This is an important finding since it questions the usefulness of an often-stated change strategy of waiting for older faculty to retire.Although new faculty bring new ideas, they also are acculturated into the existing system and are bound by the constraints of this system.
Class Size (SIZE).Class size is often mentioned by faculty as a barrier to the use of RBIS [48,56,58,59].
Here we find no support for this idea.The class size variable was only significant at the high versus low user stage.It is important to keep in mind that both groups at this stage are RBIS users.So the best conclusion about class size that can be drawn from this study is that it is not related to knowledge or use of RBISs.However, large class sizes may be a barrier to high use of RBIS (defined here as using 3 or more RBISs).
Research productivity (RSH1, RSH2, RSH3).Another characteristic that is often thought to distinguish between innovative and traditional teachers is the level of research productivity.It is often thought that faculty need to choose between focusing on research or focusing on teaching and that they cannot be highly productive in both [58,[89][90][91].Other researchers, though, have found almost no relationship between research productivity and teaching effectiveness [92,93].Our results are most consistent with the latter.Only one of our three measures of research productivity (RSH2 is the number of research publications in last two years) was significantly correlated with group membership.Having external funding for research (RSH3) and the number of research presentations made (RSH1) were not significantly correlated at any stage.The correlation with research publications is similar to that of class size in that it only holds for the high versus low use stage.Thus, like class size, the number of publications does not appear to be a barrier to RBIS use, but may be a barrier to high RBIS use.
Two other nonsignificant study variables were also related to research productivity: the type of institution (INST) and the percentage of job responsibilities related to teaching (JOB).Faculty at institutions with a physics Ph.D. (INST) and who spend less than 50% of their time on teaching (JOB) are much more likely to be productive researchers.The percentage of job responsibilities related to teaching was not significant at any stage of the innovation-decision process.The type of institution was significant at three stages in the single-variable model, but this significance disappeared for the multivariable model.Thus, our findings are similar to those of other surveys mentioned earlier that did not find differences in the use of innovative teaching between types of institutions [56][57][58].
Departmental culture (ENC, PEER).Two study variables related to departmental culture were not found to be significantly related to the use of innovative teaching.The frequency with which faculty corresponded with peers about teaching (PEER) and the level of departmental encouragement for teaching improvement (ENC) are both factors that one might expect to be significant.As noted in the limitations section, we do not think that our results related to perceived departmental encouragement are particularly meaningful since almost all respondents (92%) reported that their departments were encouraging of teaching improvement.This was an issue that we probed in a follow-up interview study with a subset of the survey respondents.Our preliminary analysis of the interviews suggest that faculty rate their departments as very encouraging of teaching improvements if the departments do not actively deter teaching innovations.For example, when asked to describe how their departments supported teaching innovations many faculty responded that they felt that they had academic freedom to use any instructional strategy that they wished as long as they did not ask for additional resources.When probed for concrete examples of support or encouragement, many faculty were unable to offer any.Faculty seeing a lack of impedance as encouragement is a phenomena that requires additional investigation.
The frequency with which faculty correspond with their peers about teaching (PEER) was also not found to be significant at any stage of the innovation-decision process.On the surface, this result does not seem consistent with recommendations towards building a culture of teaching improvement through discussion about teaching [65,94,95].We suspect that this apparent discrepancy arises because it is not the frequency of discussions that is important, but rather the content of the discussions that is important.For example, faculty in a department may routinely discuss class assignments and content coverage but never discuss actual teaching methods.Thus, our results suggest that change agents need to do more than simply have faculty engage in discussions about teaching.At minimum, scaffolding is necessary in order to ensure that these discussions actually address core issues related to teaching methods and alternatives to traditional lecturing.

Figures 2 -FIG. 1 .
Figures2-5show the odds ratios for each of the 10 variables that were significantly correlated with group membership in the single-variable models.Each figure shows the odds ratio based on a nominal logistic regression

TABLE I .
Twenty-four research-based instructional strategies included in the Web survey.

TABLE II .
Twenty explanatory variables used in this study.

TABLE III .
Stages in the innovation-decision process.

TABLE IV
. Significant explanatory variables at each stage in the innovation-decision process.An ''X'' indicates that the variable is significant at p ¼ 0:05 on a Pearson's chi-square test or one-way analysis of variance (ANOVA) without controlling for other correlates.A ''+'' indicates that the variable continues to be significant in a logistic regression model that controls for all 10 variables that were significant individually.Variables are ranked in terms of the number of stages for which they are significant.