Fixed and growth mindsets in physics graduate admissions

Considering the evidence that standard physics graduate admissions practices tend to exclude women and traditionally marginalized racial and ethnic groups from the discipline, we investigate (a) the characteristics of students that physics graduate admissions committee members seek to admit to their programs and (b) the practices associated with these admissions goals. The data for this investigation are interviews with 18 faculty who chair graduate admissions committees in programs that prioritize diversity in their graduate admissions practices. We find that some express elements of an implicit theory of intelligence known as a “fixed mindset,” in which intelligence is understood as an inherent capacity or ability primarily measured by standardized test scores and grades. Some also express elements of a “growth mindset,” in which intelligence is understood in terms of acquired knowledge and effort. Overall, most faculty interviewed expressed elements of both mindsets. A fixed mindset in physics graduate admissions is consistent with research identifying physics as a “brilliance-required” field, whose members tend to believe that raw, innate talent is a primary requirement for success in the discipline. Such a mindset directly affects the participation of women and some racial or ethnic groups, who are stereotyped as lacking such high-level intellectual ability.


I. INTRODUCTION
Physics awards smaller percentages of Ph.D.s to women (19%) and underrepresented ethnic or racial groups (7%) than any other field in the physical sciences, and these percentages are far lower than their presence in the college-age population and the U.S. population overall [1,2]. Underrepresentation is especially pronounced at selective universities that produce the largest numbers of doctoral degrees in physics (for example, see Fig. 1). The implications of this are far reaching, as these selective universities educate many of the future faculty and leaders of the physics community [3].
There is evidence that existing physics graduate admissions processes tend to exclude women and some racial and ethnic groups from the discipline, particularly by prioritizing standard achievement tests that disadvantage underrepresented groups [4][5][6]. Some U.S. physics departments, however, have admissions processes designed to increase the number of women and traditionally marginalized racial and ethnic groups in their doctoral programs. Since 2015, the American Physical Society (APS) has been studying physics graduate admissions practices. The goal of the present study is to learn how physics faculty evaluate students for admission to their graduate programs, especially how they evaluate members of underrepresented groups. The research questions are (1) What are the characteristics of students that physics graduate admissions committee members seek to admit to their programs? (2) What practices are associated with these admissions goals? Admitting people to graduate school involves judging the likelihood that they will successfully contribute to the research mission of the department and the field. This is in part a judgment of intellectual ability. These judgments are influenced by theories of intelligence (often implicit).
Through interviews with 18 faculty who chair graduate admissions committees, we find that some express elements of an implicit theory of intelligence known as a "fixed mindset," in which intelligence is understood as an inherent capacity or ability [7,8] primarily measured by standardized test scores and grades. Some also express elements of a "growth mindset," in which intelligence is understood in terms of acquired knowledge and effort [7,8]. Overall, most faculty interviewed express elements of both mindsets. However, our analysis shows that elements of a fixed mindset appear frequently even among faculty whose programs are striving to prioritize diversity in graduate admissions. This result is consistent with research identifying physics as a "brilliance-required" field, whose members tend to believe that raw, innate talent is a primary requirement for success in the discipline [9]. A fixed mindset in physics graduate admissions is likely to directly affect the admission of women and some racial or ethnic groups, who are stereotyped as lacking such high-level intellectual ability [9].
In what follows, we describe what is already known about graduate admissions practices, both in general and particularly in physics (Sec. II). We describe the theoretical perspective linking physics graduate admissions committee members' apparent theories of intelligence to their descriptions of their admissions goals and practices (Sec. III). We outline the data collection for this study (Sec. IV), and then articulate and apply a coding scheme to characterize the data (Sec. V). Finally, we interpret the results of our analysis of physics graduate admissions goals and practices (Sec. VI).

II. GRADUATE ADMISSIONS PRACTICES AND DIVERSITY
Across disciplines, applicant undergraduate grade point average (GPA) and standardized test scores, particularly the Graduate Record Exam (GRE), are the most important factors in admissions decision making [4,6]. Many physics departments prioritize undergraduate GPA and the physics subject GRE (more than the general GRE), and also evaluate an applicant's research record, usually through letters of recommendation from faculty supervisors of undergraduate research experiences [10]. Admissions committees may score each of these components quantitatively, then combine these subscores into an overall score. Other committees assign applicants a single overall score (e.g., from 1 to 5). In either case, committees usually use these scores to rank their applicants in order of desirability.
Many admissions committee members make conscious efforts to include members of underrepresented groups in the accepted class [11,12], which for the purposes of this study includes African Americans, Hispanic Americans, and Native Americans. However, most graduate programs consider diversity after the applicant pool has already been filtered by factors that disproportionately rule out female, African American, Hispanic American, and Native American applicants [4,6,[13][14][15]: in particular, GRE scores are systematically lower for members of underrepresented racial or ethnic groups and women [5,16].
Some institutions attempt to replace practices that they consider exclusionary with practices that are designed to include more students that do not meet traditional criteria. The term "holistic admissions" refers to admissions decisions based on several different factors, none of which alone will automatically cause an applicant to be accepted or rejected [17]. It may also refer to admissions processes that emphasize the whole person, focus on interviews and personal statements, prioritize nonacademic experiences such as work and leadership roles, or emphasize noncognitive skills such as critical thinking, grit, persistence, and creativity [18][19][20][21][22]. A holistic admissions approach can help institutions avoid complications that can be associated with affirmative action. (Federal law permits the inclusion of race as a factor in university admissions, but some states do not, and admissions decisions cannot be based solely on race or gender [23,24].)

III. FIXED AND GROWTH MINDSETS
Academic theories of intelligence mainly debate whether intelligence consists of a single property or multiple independent factors [25][26][27][28]. Popular debates about the nature of intelligence sometimes include arguments that imply racial differences in intelligence [29] or attribute differences in standardized test performance to broad social inequities [5,30]. Research on fixed mindset and growth mindset, summarized below, offers a perspective on the relationship between an individual's apparent theory of intelligence and that person's description of their decisionmaking behavior. This theoretical perspective supports the articulation of a coding scheme for analyzing this study's faculty interviews, described in the next section. Specific research findings from the literature correspond to particular codes in the coding scheme. A. Fixed and growth mindsets about oneself A person's convictions about the nature of intelligence inform their self-judgments and influence their behavior. Dweck has identified two primary implicit theories of intelligence that people tend to hold, termed fixed mindset and growth mindset [7,8,31]. Those with a fixed mindset believe that each person has a fixed quantity of intelligence; they interpret difficult cognitive tasks or academic settings as potentially revealing the limits of their intelligence, and therefore may choose to avoid them [7,8,31]. Alternatively, those with a growth mindset believe that intelligence is a capacity that can change incrementally with increased knowledge and effort; they interpret difficult intellectual tasks as opportunities for learning, and may seek them out [7,8,31]. Dweck presents evidence that people tend to hold one or the other mindset consistently in many situations over years, and that a fixed mindset is common in the U.S. [7,8]. (Our analysis, presented in Sec. V, includes evidence that a single individual may represent a variety of convictions, corresponding to both fixed and growth mindsets.) The fixed mindset, also called the "entity theory of intelligence," is the idea that intelligence is a fixed entity-a quality that is inherent or innate to a person and not likely to change with development or experience [7,8]. A person with a fixed mindset tends to agree with statements such as, "Your intelligence is something very basic about you that you can't change very much," or "You can learn new things, but you can't really change how intelligent you are." Extensive research demonstrates that people with a fixed mindset tend to cope poorly with setbacks on tasks that they perceive to require intelligence, because failure is understood as evidence that their intelligence is (permanently) low [7,8]. In this mindset, people feel "smart" when they can perform challenging tasks easily; high effort is judged negatively because it suggests lower ability [32][33][34]. For example, those with a fixed mindset judge someone as "more intelligent" who does not study for a test, compared to someone who studies hard, even if the hardworking student scores higher-reasoning that if you must work hard for a result, you must not be as smart.
An alternative conviction is that "no matter how much intelligence you have, you can always change it quite a bit" [7,8]: this is called a "growth mindset" or an "incremental theory of intelligence." People with a growth mindset readily engage and persist in intellectually challenging tasks, because such tasks are seen as learning opportunities [35]. In this mindset, people feel smart by "engaging fully with new tasks, exerting effort to master something, stretching their skills, and putting their knowledge to good use" [8]. High effort and persistence are judged positively [32][33][34].
Those with a fixed mindset tend to judge themselves rapidly based on a small number of ambiguous experiences [8,36]. If they miss an appointment, get a low grade on a test, or are rebuked by a friend, they may think, "I'm a loser"; they tend to interpret their experiences as telling them how good (or bad) they are. Conversely, a person with a growth mindset who experiences the same events might think, "I didn't check my calendar," "Perhaps I should have studied more," or "My friend may have been having a bad day." They do not necessarily attribute bad experiences to their own failings, and interpret challenging experiences as opportunities to learn.

B. Fixed and growth mindsets about others
Our first research question ("What are the characteristics of students that physics graduate admissions committee members seek to admit to their programs?") is concerned with judging others. Those with a fixed mindset judge others' intelligence to be innate, rather than acquired through earlier life experiences (just as they judge their own intelligence to be innate). Asked about babies switched at birth, those with a fixed mindset tend to say that a baby born to parents "who are not so smart" will always be "not so smart," reasoning that the baby will innately have a low IQ even if the baby is raised by "smart" parents. Those with a growth mindset, in contrast, say that the baby will likely grow up "smart" because it will grow up with people who encourage learning [8]. Physics, in particular, is a field "whose members cherish brilliance"who tend to believe that raw, innate talent is a primary requirement for success in their field [9]. This mindset may directly affect the participation of women and underrepresented racial or ethnic groups, who are stereotyped as lacking such high-level intellectual ability [9].
A fixed mindset is associated with valuing prior achievement as evidence of high intelligence, especially standard measures of achievement such as IQ, grades, the Scholastic Aptitude Test (SAT), and the Graduate Record Exam (GRE) [8]. Such standard measures of achievement are traditional in graduate admissions [4,13,37,38]. An indepth investigation of graduate admissions practices by Posselt found that at least half of the faculty in the study "instinctively associated GRE scores with intelligence": for example, one sociology faculty member said, "GREs tell me something probably, about-I don't know-crude about native intelligence or general intellectual horsepower or something like that" [6].
A growth mindset, in contrast, is associated with valuing characteristics of students that can support intellectual growth, such as creativity, curiosity, energy, perseverance, and passion for the subject matter [2]. Programs that identify as using "holistic" admissions tend to emphasize these characteristics, often termed "noncognitive factors" [18][19][20][21][22], and graduate admissions faculty may attend to these qualities when considering an applicant's research experience [6]. For example, in Posselt's study a faculty member (in astrophysics) negatively evaluated an applicant because there was "no evidence of creativity or any synonyms for it in the letters" of recommendation [6].
Research demonstrates that those with a fixed mindset judge others rapidly, on the basis of just a few observed behaviors or incidents [8]. They make both positive and negative judgments in this way. This rapid judgment is not merely expedient, but stems from a belief that character is "a unitary thing that permeates virtually all actions and displays itself with great consistency" [8]. According to this research, those with a fixed mindset "do not think they are making unwarranted snap judgments. They think they are reading others' traits from reliable information." After forming an initial impression, those with a fixed mindset are less sensitive to new information that contradicts their initial impression and may even try to avoid it [8]. This quality is potentially relevant to graduate admissions, where faculty often find themselves in the position of making rapid, high-stakes decisions about applicants based on limited information. Finally, those with a fixed mindset exhibit higher levels of stereotyping of various social groups. This is the case for both positive stereotypes and negative ones, and for existing social groups (such as ethnic groups) as well as novel groups (about which people have just learned) [39,40]. For example, children as young as six show gender stereotypes as to who is "really, really smart" [41]. Research suggests that this is not necessarily attributable to bigotry, but rather indicates a straightforward belief that "what they perceive on the outside reflects what people are like on the inside" [8]. Those with a fixed mindset believe not only that traits such as intelligence are innate, but also that one can judge such traits readily from small samples of overt behavior (whether their own, or others') [39,40]. These rapid judgments of groups of people on the basis of a few actions or measures emerge as stereotypes, which operate in graduate admissions practices as in other social settings [4,6].

C. Detecting mindsets in research
Research about fixed and growth mindsets traditionally categorizes people based on their responses to a "mindset questionnaire," in which people agree or disagree with direct statements such as "You have a certain amount of intelligence, and you can't really do much to change it" [7,8]. This method rests on the conviction that people have a theory of intelligence that is unitary (either all fixed or all growth, rather than a mix of different elements) and stable (persisting over time), perhaps implicit but readily available upon prompting. In support of this unitarity and stability, Dweck presents evidence that people answer the mindset questionnaire consistently (suggesting unitarity) and that they do so in many situations over years (suggesting stability). However, she also presents evidence that mindset can change, and that growth mindset can be taught through relatively brief interventions [8]. Characterization of theories of intelligence as unitary and stable may be influenced by the fact that the mindset questionnaire asks directly about "intelligence," which may be understood as a context-independent quality.

IV. DATA COLLECTION
The study presented in this paper is associated with the American Physical Society (APS) Bridge Program, whose goal is to increase the number of physics Ph.D.s awarded to students belonging to specific racial or ethnic groups (in this project, African Americans, Hispanic Americans and Native Americans) [42]. The APS Bridge Program has created sustainable transition ("bridge") programs and a national network of graduate programs that provide substantial support for students to successfully complete Ph.D. programs.
Through its national network, the APS Bridge Program has previously observed that some physics departments are attempting to move toward greater inclusion by changing their admissions practices to admit more students from underrepresented groups. Data for this study were collected with the intention of documenting graduate admissions practices that prioritize diversity.

A. Sample
Participating departments met all of the following criteria: • The department prioritizes diversity in its graduate admissions practices, as indicated by its reputation with the APS Department of Education and Diversity or the results of a graduate admissions survey [10]. • The department is physics or a closely related discipline (e.g., applied physics). One diversity-oriented math department was also included in the study due to its history of efforts to diversify graduate mathematics. • The department admits less than 20% of all applicants, meaning that there are substantive choices made in the admissions process. All departments that were invited to participate did so (there were no self-selection effects). Most of the participating departments were doctoral institutions with "highest research activity," according to the Carnegie classification of institutions of higher education. An interviewer (Scherr) conducted hour-long interviews with personnel from participating departments using the interview protocol outlined below. Interviewees had (i) been on the physics graduate admissions committee or significantly involved in the admissions process in the last year, and (ii) experienced at least one full iteration of their department's application process in the last year, including engaging in the entire decision-making process that led to offers being made. Interviews were conducted by telephone and documented with extensive notes. (The interviews were not audio recorded because of concerns about self-censoring.) In total, eighteen faculty members from sixteen departments were interviewed. In two cases, two people from a single department were interviewed-the current and former graduate admissions chair-to gain a fuller picture of the admissions process for that department. The interview protocol included questions designed to detect whether there were competing perspectives on the admissions process.

B. Interview protocol
The interview protocol is reproduced in Fig. 2. Most of the interview was spent on the first question, which invited interviewees to describe their department's admissions processes in detail. This was asked as an open-ended question to permit interviewees to describe whatever they understand to be important in the admissions process. The primary goal was to obtain a detailed description of the mechanics of the admissions process, along with the motivation, purpose, and mindset informing those practices. Questions 1, 2, and 3 (in bold font in Fig. 2) were asked in every interview. Other questions (including the many bulleted questions listed under question 1) were not asked in every interview; rather, they were used as optional prompts to bring out the most detailed possible picture of the admissions process. The interviewer (Scherr) is a white female faculty researcher who conducts research on how departments promote diversity and inclusion in physics.
This interview protocol was intended to elicit descriptions of each department's current admissions process, not each interviewee's personal convictions about the optimal approach to physics graduate admissions. That said, some physics departments do not have a well-defined admissions process, and in many cases physics faculty who serve on a graduate admissions committee have substantial freedom in how they evaluate applicants. In this common situation, departmental practices are something of an emergent property, arising from the combined individual commitments and behaviors of the faculty who serve on the admissions committee. For this reason, it can be difficult to separate an individual's description of the department's practices from his or her own views about how applicants

Describe your admissions process.
•  In what follows, we refer to the admissions practices of programs as represented by individual faculty members, acknowledging that there is often not a clear distinction between the faculty member's perspective and the approach taken by the program as a whole.

V. ANALYSIS
For our study, we did not assume that implicit theories of intelligence are unitary. Instead, we allowed for the possibility that a single participant might refer to a variety of convictions or practices, corresponding to elements of either fixed or growth mindsets. To this end we deductively created a coding scheme based on the many specific findings about fixed and growth mindsets described above, and coded the notes from the faculty interviews using this scheme. In what follows we illustrate the coding scheme, demonstrate interrater reliability, present the results, and discuss limitations of the analysis.

A. Coding scheme
To prepare the notes for analysis, identifying information was removed and interviewees were assigned alphabetic pseudonyms (Anderson, Brooks, Carter, etc.). Table I presents eight codes associated with a fixed mindset (F1-F8) and Table II presents eight codes associated with a growth mindset (G1-G8). These codes are based on the mindset research described in Sec. III and are illustrated with example statements from the interviews that received each code. (Some codes did not appear in this data corpus and are illustrated with examples from the research instead.) "Find the diamond… the obvious cream of the crop" (Anderson) "Blue-chip students" (Edwards) "The top people" (Garcia) "The best individuals" (Hill) "This person is a research genius" (Jones) "Brilliance…the real stars" (Morris)

F2
Negative judgment of high effort (you have higher ability if things come easily to you) [32][33][34] "If you have to work hard on some problems, you're probably not very good at them." [8] F3 Attribution of students' shortcomings to personal deficiencies (e.g., they have not "got it") [8] "If I got a low GRE score I would think I was dumb." [36] F4 Attribution of failure to students' shortcomings (e.g., those who drop out were unqualified in the first place and should not have been admitted) [8] "Women who are not finishing the program, it's not because the program failed them, other than maybe admitting them when they should not have been admitted." (Edwards) "90% of them succeed once they get in, so we must be admitting the right ones." (Landon) "People don't drop out because they're failing the classes-that's what 'good' means." (Morris)

F5
Stereotyping of applicants from underrepresented groups (e.g., statements that underrepresented groups are less interested in physics or less prepared for it) [39,40] "In the top 25% we look for women who are as good as the other people." (Brooks) "If the Indian students do well then next year we let more Indian students in." (Fong) Fixed mindset practices F6 Rapid, certain judgments of others [8,36] "Most of the cases are straightforward because it all goes together…Students with high GREs are strong students all around." (Edwards)

F7
Judgments of others that focus on prior achievement, rather than potential [8] "We want basically 'A' students in physics courses." (Fong) "GPA and GRE." (many)

F8
Value placed primarily on standard measures of achievement (GRE and grades) [8] (similar to F7) TABLE II. Coding scheme for growth mindset (G1-G8), with sample text from interviews to illustrate each code. Codes G1-G5 refer to beliefs or attitudes associated with the growth mindset; codes G6-G8 refer to practices associated with that mindset. Names are pseudonyms of individual interviewees.

Code Description Sample
Growth mindset beliefs G1 Statements implying expectations of intellectual/ academic growth; attention on the amount of knowledge one possesses and how one uses it [7,8,31] "If their grades show a positive trajectory, that's great… we are looking for a sense of progress." (Iverson) "Success is not just good grades but also long term, will they make us proud down the road…There is an understanding that we are making a lifetime commitment to them." (Norton)

G2
Positive judgment of high effort (you have higher ability if you work hard) [32][33][34] "When something comes easily to you, you don't know how good you are at it." [8] G3 Attribution of students' shortcomings to external circumstances (e.g., they did not have access to good courses) [8] "Applicants from HBCUs may not have had the opportunity to take two full semesters of intermediate quantum." (Iverson) "They may be coming from a background that hasn't made it totally clear what their potential is-small school, family problems, working 30 hours a week to get through school." (Ortega)

G4
Attribution of failure to department's shortcomings (e.g., those who drop out were poorly served by the program) [8] "The purpose is to admit students who will thrive under certain very good conditions that the department is responsible for creating." (Donaldson)

G5
Conscious aversion to stereotyping of applicants from underrepresented groups (e.g., statements that physics talent is uniformly distributed across race and gender, statements that differences in groups' preparation or achievement is due to social circumstances) "If you have a 'subliminal GRE cutoff' you will eliminate all the underrepresented minorities…We do interviews remotely, audio-only, to reduce unconscious bias." (Carter) "We are looking for students who for one reason or another are remote from the profession. We would like to bring them in and… increase the number of women and minorities who enter the workforce." (Donaldson) "We know the GRE discriminates against women and URMs." (Jones) Growth mindset practices G6 Complex, provisional judgments of others [8,36] "Sometimes the situation was way more complicated than the score would indicate." (Carter) "If there is a student with bad GRE scores but good grades, we call the faculty mentor… It took time to build those relationships but it's worth it." (Donaldson)

G7
Judgments of others that focus on potential, rather than prior achievement [8] "Can this student be successful in our graduate program with the kinds of support we have available for this student?" (Iverson) "We have to get away from thinking that admission [to our university] is a prize [for undergraduate achievement]… Our goal is not recognizing achievement, it is recognizing potential." (Kim) "Students that potentially have great talent but have not expressed that in a way that it is most obvious to an admissions committee." (Ortega) G8 Value placed on grit or noncognitive factors (passion for physics, determination, perseverance, coping with adversity) [8] "Self-motivation, ability to work independently, true interest in their field-really inspired by it." (Carter) "Independence, grit, writing-qualities we want to look for in the applicant, instead of documents." (Rourke) In each category, the first five codes (F1-F5 and G1-G5) refer to beliefs or attitudes associated with a particular mindset, such as the idea that intelligence is an inherent or fixed quality (F1) or an expectation of intellectual growth (G1). These codes are intended to provide evidence for responding to the first of our two research questions: "What are the characteristics of students that physics graduate admissions committee members seek to admit to their programs?" The other three codes in each category (F6-F8 and G6-G8) refer to practices associated with a particular mindset, such as rapid, certain judgments of others (F6) or complex, provisional judgments of others (G6). These codes are intended to provide evidence for responding to our second research question: "What practices are associated with these admissions goals?" To illustrate how some of these codes reflect fixed and growth mindsets, we reiterate a basic difference between the mindsets: Those with a fixed mindset interpret failures as reflecting personal shortcomings, whereas those with a growth mindset understand failures in terms of unfavorable circumstances in the environment. One way that this difference manifests in graduate admissions is in how a graduate admissions committee member views shortcomings that might be evident in students' application materials: they might attribute students' shortcomings to personal deficiencies (e.g., they have not "got it"; code F3) or to external circumstances (e.g., they did not have access to good courses; code G3). Another way that this difference can manifest is in how a graduate admissions committee member understands failures that occur during graduate school: they might attribute failure to students' shortcomings (e.g., those who drop out were unqualified in the first place and should not have been admitted; code F4) or to a department's shortcomings (e.g., those who drop out were poorly mentored and took the wrong classes; code G4). These codes, and the others, express codes suggested by the mindset literature in terms specific to graduate admissions.
In using this coding scheme, we aim to detect signals of people's mindset that they broadcast, perhaps unconsciously, as they use natural language to describe their professional activity. Our approach is significantly different from widely cited mindset research [7,8], in which participants respond directly to a well-specified prompt on a questionnaire. However, both methods are broadly consistent with a sociocultural theory [43][44][45][46][47][48], addressing "how people's beliefs, values, and goals set up a meaning system within which they define themselves and operate" [8].

B. Reliability
To assess the reliability of the coding scheme presented in Tables I and II, two researchers (Scherr and Gray) coded the interview data independently. The coding unit was the paragraphs that the interviewer had created to mark transitions in the interview discourse. For each paragraph, each coder assigned as many of the 16 codes presented in Tables I and II (F1-F8, G1-G8) as were appropriate to that paragraph. Two randomly selected interviews (pseudonyms Anderson and Brooks) were used for calibration between coders. For the other interviews, agreement was assessed for both the codability (the extent to which coders agreed that each given paragraph should be associated with one or more of the 16 codes) and the codes (the extent to which coders agreed on the specific codes to be associated with that paragraph). The reliability of codability was assessed as a ratio of the number of paragraphs that coders either both coded or both did not code to the total number of paragraphs. The reliability of codes was assessed according to the following expression: ½ð# possible codes for all codedparagraphsÞ − ð# disagreements in all coded paragraphsÞ=# possible codes for coded paragraphs which may also be expressed as follows: ½ð# possible codesÞ × ð# coded paragraphsÞ − ð# disagreements in all coded paragraphsÞ=½ð# possible codesÞ × ð# coded paragraphsÞ.
The number of possible codes is 16 (F1-F8, G1-G8). The number of disagreements associated with a paragraph was counted as the total number of codes that were given by only one coder rather than both: for example, if one coder assigned (F1, G1, G3) and the other assigned (F1, G2), this was counted as three disagreements.
Interrater reliability was consistently high, with codability typically above 80% and codes always above 90%. Table III displays both kinds of interrater reliability for each interview. Preliminary coding disagreements were always resolved after discussion. Table III shows the number of times each code was associated with each interview, along with the interrater reliability (for codability and codes) in each case. Codes F2 and G2 did not appear in this data set, and are not included in Table III.

D. Limitations
One weakness of the study design is that because interviews were not recorded (for fear of self-censoring), the primary data are notes made during the interviews, not transcript. Such notes are potentially influenced by the researcher's theoretical perspective and implicit assumptions, even though they were taken before the current theoretical perspective was articulated or the coding scheme was developed. For example, it is possible that the faculty member who claimed to be seeking "the top people" (Garcia) could have been referring to applicants with outstanding research experience (or some other "top" quality), rather than those with the highest intellectual potential.
The coding scheme, though demonstrated to be reliable, also has limitations. As a deductive coding scheme, it imposes a particular theoretical perspective on the data (that of fixed and growth mindsets), rather than allowing participants' own meanings to generate the primary themes [49]. In practice, coders found it difficult to distinguish some codes (particularly F7 and F8, both of which refer to reliance on grades and standardized test scores as measures of prior achievement), and some codes did not appear in this data corpus (F2 and G2). It is almost certainly inappropriate to add the frequencies of various codes to get an overall score (e.g., to evaluate Edwards as getting 18 points for fixed mindset and 4 for growth mindset), since the items in the coding scheme are not independent or of equal weight.

VI. DISCUSSION
In what follows, we interpret the results of the analysis presented in Sec. V.

A. Characteristics of students that programs seek to admit
The primary aim of this research was to investigate (a) the characteristics of students that physics graduate admissions committee members seek to admit to their programs and (b) the practices associated with these admissions goals. The results suggest that some programs seek to admit students who are judged to have innate talent, primarily measured by grades and standardized test scores (especially the physics GRE). This approach negatively impacts the admission of women and some racial or ethnic groups, who perform lower on standardized measures of achievement [4][5][6]16] and are stereotyped as lacking high-level intellectual ability [9]. Some programs, in contrast, seek to admit students who are judged to have the potential for growth, partly measured in terms of noncognitive factors. This approach has improved diversity in other graduate fields [18][19][20][21][22].

B. Lack of unitary theory of intelligence
All the interviewees expressed elements of a growth mindset, and almost all expressed elements of a fixed mindset. This result is evidence against the idea that the beliefs and practices associated with individuals in this study, and/or their physics departments, comprise a unitary theory of intelligence.

C. Overall fixed or growth mindset
Even though nearly all interviews express elements of both fixed and growth mindsets, 12 of the 16 interviews analyzed were associated with a preponderance of codes indicating either fixed or growth mindset. Edwards, Fong, Langdon, and Morris had many codes associated with a fixed mindset and relatively few associated with a growth mindset. Carter, Donaldson, Iverson, Kim, Norton, Ortega, Quinn, and Rourke had several codes associated with a growth mindset and fewer associated with a fixed mindset. In other words, half the faculty in this data set indicate that their department has (or they have) an overall growth mindset regarding graduate admissions, and one-fourth TABLE III. Results of coding interviews with graduate admissions committee members for fixed and growth mindset, documenting the frequency of each code (F1, F2, etc.) for each interviewee (Carter, Donaldson, etc.). Inter-rater reliability is presented for both codability (the extent to which coders agreed that each given paragraph should be associated with one or more of the codes) and codes (the extent to which coders agreed on the specific codes to be associated with that paragraph). indicate a predominantly fixed mindset. The remaining four interviews were associated with a substantial mix of codes associated with both fixed and growth mindsets, such that a clear overall mindset is not indicated. As stated in Sec. IV, departments that participated in this study were chosen because they were understood informally to be prioritizing diversity in their graduate admissions practices, in an environment of substantial admissions pressure. Thus, we do not expect that the results in Table III are representative of graduate programs in general. Instead, Table III suggests that in these selective graduate programs that are striving to prioritize diversity, about half are described by their faculty as having an overall growth mindset and one-fourth as having an overall fixed mindset.

D. Belief in innate talent
The commonest code was F1, indicating that statements of innate ability, or "smart talk" [6], are common among the faculty interviewed. Codes F3 and F5 were relatively rare, suggesting that faculty in this data set rarely state negative judgments of individual students' innate capacity (F3) or express racial or gender stereotypes explicitly (F5). The implication is that the most common behavior associated with bias in admissions is not explicit stereotypes but rather a belief in innate talent.

E. Association between beliefs and practices
Codes F1-F5 refer to fixed-mindset beliefs or attitudes expressed by the interviewee (e.g., that intelligence is innate, or that those who drop out were unqualified in the first place), while F6-F8 refer to fixed-mindset admissions practices (such as heavy reliance on the GRE). Interviews that had many of the F1-F5 codes tend to also have many F6-F8 codes: in other words, those who described beliefs associated with a fixed mindset also tended to describe practices associated with a fixed mindset. For example, Edwards described both fixedmindset beliefs-describing students her program would definitely admit as "blue-chip students" (referring metaphorically to the most valuable poker chips; code F1), and asserting that women who did not finish the program were probably wrongly admitted in the first place (code F4)and fixed-mindset practices such as rapid, certain judgments (code F6).
Similarly, codes G1-G5 refer to beliefs or attitudes associated with a growth mindset (e.g., an expectation of academic development), and codes G6-G8 refer to admissions practices associated with a growth mindset (e.g., prioritizing noncognitive factors such as persistence and creativity). Interviews with many of the G1-G5 codes tend to also have many G6-G8 codes-and relatively few F codes. Iverson, for example, describes both growth-mindset beliefs (e.g., attributing students' shortcomings to external circumstances such as lack of access to good courses; code G3) and growth-mindset practices (e.g., judging students in terms of their potential rather than their prior achievement; code G7).
An implication of this analysis is that making graduate admissions more inclusive may entail not only shifting programs' practices to be more aligned with a growth mindset, but also shifting the associated beliefs and attitudes.

F. Unexamined goals for graduate admissions
Many physics departments do not have explicitly stated goals for their graduate admissions process. When prompted to reflect on their goals, most state that their aim is to admit students who will thrive in their department, primarily meaning that they will pass the qualifying examination and conduct successful research. Edwards stated this goal as, "Strengthen our research by admitting people who will be great at it." Most physics graduate admissions committee chairs interviewed for this study reported being unaware of other departments' practices, and were interested to learn whether other departments had better approaches or had solved problems they also faced. This suggests that departments might be willing to change their practices if they become aware of alternatives, understand the impacts of these practices, and have evidence for the success of alternative approaches. This study suggests that departments wanting to move toward more inclusive graduate admissions processes should explicitly and transparently identify primary goals of the physics graduate admissions process and the corresponding qualities of desirable applicants, along with appropriate metrics for the success of the physics graduate admissions process.

VII. CONCLUSION
Our results suggest that elements of both fixed and growth mindsets are common in physics graduate admissions committee members. Our analysis includes evidence that a single individual may represent a variety of convictions, corresponding to both fixed and growth mindsets. Significantly, elements of a fixed mindset persist even among those whose departments are striving to prioritize diversity in admissions. This fixed mindset manifests in both beliefs about who is likely to succeed in physicsthose with innate talent-and admissions practices that emphasize undergraduate grades and physics GRE scores over other possible measures. Both these beliefs and these practices tend to exclude women and some racial or ethnic groups from the discipline [4][5][6]9]. Our findings are consistent with theory and research in fixed and growth mindset as well as research in graduate admissions in other disciplines; the contribution of this research is to illustrate these findings in the specific physics disciplinary context, which is notable for its lack of diversity [1,2] and emphasis on innate talent [9]. Fortunately, our results also document a growth mindset among some graduate admissions committee members, including both growth-oriented beliefs (e.g., that students can grow into physics excellence with support) and growth-oriented practices (e.g., placing value on noncognitive factors). Programs that apply a growth mindset to their admissions processes include APS Bridge Programs associated with highly competitive physics departments, which are documented to have a high rate of student retention [42]. These programs embrace and promote cultural change, anticipating benefit to the field, while remaining committed to the academic integrity that has proven so powerful for physics in solving some of the most complex problems in the universe. As one interviewee (Donaldson) stated: "We have to be the example of showing that nothing bad happens when you change who does the science."