Admissions practices in terminal master ’ s degree-granting physics departments : A comparative analysis

There has been little work done in exploring the variations in admissions practices between different types of physics graduate programs. In this paper, we compared admissions practices in master’s programs to a parallel data set collected from doctoral departments to understand the relative importance of particular admission criteria and looked especially at similar sized and “pressured” programs (as measured by the ratio of applications received-to-intended cohort size). We found that master’s programs experience notably lower application pressure compared to doctoral programs overall. In the prioritization of admission criteria, undergraduate course-taking and undergraduate math and physics GPA received the shared highest priority, while recommendation letters and Graduate Record Examination (GRE) quantitative scores are the next important criteria but are statistically less important for master’s programs in comparison with doctoral programs. In the targeted comparison of similarly sized and pressured master’s and doctoral programs, the higher importance of GRE quantitative scores and the lower importance of prior research considerations in master’s programs persist. We argue that the relatively lower importance placed on the quantitative GRE and physics subject GRE scores enhances the likelihood of success for undergraduates applying to master’s programs, particularly for those who did not take or score well on the quantitative GRE and physics subject GRE test, which have been found to incorporate significant biases against many students but are nonetheless normative in many admissions decisions. The research and graduate level course experiences gained through a master’s program may improve postgraduate job opportunities or even continuation on to doctoral research. However, the shared importance of certain factors in admissions between master’s and doctoral programs risks propagating a systematic inequity.


I. INTRODUCTION
As science and engineering job markets expand globally, the need for highly skilled physicists is increasing.Postgraduate education in physics opens a greater array of career opportunities to students than those available after an undergraduate degree or high school physics.Physicists with a master's or doctoral degree in physics report a higher income and more intellectually stimulating opportunities [1].Students desiring to pursue such a career can earn a master's or Ph.D. degree.Those who are successful in completing a postgraduate degree in physics gain access to these improved personal opportunities but also gain influence over the current and future agenda of the physics community; admissions processes play a critical role in deciding who forms the annual cohort of graduate students, and thereby get the opportunity to pursue postgraduate physics careers.
Recent literature in graduate admission practices has repeatedly flagged concerns about certain admission practices [2][3][4][5][6].In particular, Graduate Record Examination (GRE) scores have been reported to be weighted too heavily in doctoral admissions [7], including their use as a minimum cutoff by many programs [8].GRE scores have been found to discriminate against students from underrepresented groups in physics which limit their opportunities in graduate education [4].Some studies report similar issues in other science disciplines [9].In other work, GRE scores have been found to correlate weakly-to-poorly with students' abilities [10,9,5].More recently, alternate ways of evaluating students' scientific potential by focusing on grit, interest, and college and research experiences have been discussed [11][12][13].In another recent study of graduate physics departments, researchers indicated that "growthoriented" practices (e.g., early and sustained mentoring relationships with faculty members [14]) improve circumstances for graduate students' entry and progress in some graduate programs.However, as these practices are relatively rare, it remains critical to identify the primary practices of admissions in physics departments [6].
While the aforementioned work has broadened our understanding of current graduate admission practices, one limitation has been a primary focus on doctoral-degree granting programs; terminal master's degree-granting programs are understudied in general, especially in terms of admission practices.Students seeking admissions to graduate school may perceive no particular distinction between the admission process of these two types of graduate programs, though this may not be the case.Moreover, students may be limited in their opportunities if they apply to departments facing high "admission pressure" (e.g., programs receiving a large number of graduate applications compared to their intended cohort size), or do not take into consideration the differences in practices for different types of programs (e.g., master's versus doctoral programs, etc.).However, to date, there is also no clear evidence on the distinctiveness of admission practices for master's programs.In the current paper, we deliberately focus on exploring the differences in admission practices amongst terminal master's and doctoral programs, and how admission pressure mediates any difference.
Students interested in pursuing a master's degree in physics in particular will benefit from an improved understanding of how their applications are evaluated.The similarities and differences in master's programs compared to the doctoral programs can better inform prospective students.Further, having an understanding of these differences, physics faculty can more effectively mentor students about their postgraduate opportunities.
Finally, graduate physics departments are one of the least diverse science communities [15].The critical role played by master's program admissions has been of interest for the American Physical Society's (APS) efforts to improve graduate diversity.Prior research has indicated that students from underrepresented groups in science, technology, engineering, and math (STEM) are 50% more likely to select master's degree en route to a doctoral degree (compared to majority students) [16], thus emphasizing the relative importance of master's programs towards doctoral degree success and diversification.Therefore, how master's programs select their cohort has significant importance in understanding diversity in graduate education.

II. TYPES OF PHYSICS MASTER'S DEGREES
Physics master's degrees are commonly awarded in two different modalities in the United States; those awarded en route to a Ph.D., and "exiting" or terminal master's degrees.The former is often a provisional degree awarded to graduates who typically continue their doctoral research in the physics discipline at the same department.The latter is awarded when the student leaves their current department either by design (a "terminal degree") or by virtue of not progressing through a Ph.D. program (an "exiting" degree).Some master's degree recipients do their doctoral research at the same university with a different (nonphysics) department or at a new institution while some go on to work as postgraduate professionals.
Amongst the physics graduate population, terminal master's degree recipients represent about one-third of degrees awarded, as of 2011 [17].This represents a total of 801 students who received a physics master's degree in the year 2011-2012 Looking at the distribution across departments, for a terminal master's degree in physics, in 2011, about 84% of master's degrees are awarded by Ph.D. granting institutions, 14% are awarded by master's granting institutions, with the remaining 1% awarded by other specialized research centers or institutes [18].For physics alone, in 2011, 62 departments offered an exiting master's degree as the highest degree and 195 Ph.D.-granting departments offered a master's degree in physics.Moreover, about two-thrids of terminal master's programs in physics have a specific research focus as part of their degree requirements [17].For holders of a terminal master's degree in physics, a dominant trend has been working in STEM-related industries, with engineering (45%) and computer and information technology (24%) being the most common fields [19].

III. UNDERREPRESENTATION TRENDS IN PHYSICS MASTER'S PROGRAMS
The American Physical Society identifies African American, Hispanic American, and Native American/ Alaskan Native populations as underrepresented in physics.Only about 3% (n ∼ 27) and 2% (n ∼ 18) of master's degrees were obtained by Hispanic Americans, and African Americans, respectively, in the period of 2010-2012 [17].By comparison to the fraction of the college-age population in the U.S.-21% and 16%, respectively-master's degree recipients from these underrepresented groups are exceptionally low in numbers [20].In particular, the underrepresentation of African American-identified students in graduate education is reflective of the long-term negative trend at the undergraduate level as well [18,21].
Historically black college and universities (HBCUs) award about one-half of all master's degrees received by African American students [17].Programs like the Fisk-Vanderbilt Master's-to-Ph.D.-Bridge and APS Bridge program have developed successful, structured efforts to help students transition to a Ph.D. program [12,14,15,22].Considering the dearth of underrepresented students in graduate physics, more intense efforts for the recruitment and admissions of such students are called for.

IV. RESEARCH QUESTIONS
In the previous sections, we highlighted the current gaps in understanding terminal master's degree-granting admissions practices.Understanding graduate admission practices better can frame the discussion of who can apply, who gets considered, and who will form the future community of physicists.An improved understanding of admission practices, and how these match (or differ) with doctoral admissions practices will benefit both prospective students and physics departments.Further, an understanding of how these norms compare between master's and Ph.D. programs can inform students to make better choices of graduate programs.This study aims to investigate the similarities and differences in admission practices between master's and doctoral programs according to 20 graduate admissions criteria.Thus, the research questions guiding this study are as follows: Research question 1: What is the application pressure faced by terminal master's degree-granting physics departments, by comparison to doctoral-degree granting departments?Research question 2: How do the use and prioritization of various admissions criteria (including prior course taking, GRE scores, prior research experiences, and other factors) compare between master's and doctoraldegree granting departments (considering all programs regardless of size and pressure)?Research question 3: How do similarly sized and pressured master's and doctoral programs compare in their prioritization and use of various admissions criteria?

V. SURVEY DEVELOPMENT AND DATA COLLECTION
In this Letter, we address these research questions using data from a graduate admissions survey developed to explore current admission practices amongst physics departments that award a master's degree as its highest graduate degree.This survey was developed as a research component of the APS Bridge Program, an NSF-funded multiyear program focused on increasing diversity in graduate education in physics [23].
An initial version of the survey was developed for the doctoral-degree granting programs and was tested for construct and content validity at the 2nd Graduate Education Conference at College Park, Maryland in 2013.The final version involved 30 multipart questions.Some questions used an anchored scale, others were open ended.After developing the survey for Ph.D.-granting departments (discussed elsewhere [8]), it was redeveloped to focus on physics departments offering exiting master's as the highest award.Both surveys were very similar in design, except for minimal wording changes to contextualize the survey for the program type.The final version of the master's survey appears as Supplemental Material [24]; the final version of the doctoral survey appears in Ref. [8].
To gather responses, we used a national list maintained by the APS of the department chairs and/or directors of graduate admissions for master's degree-granting programs.These contacts were also requested to forward the survey to other faculty members involved in graduate admission decisions, and periodic reminders were sent out in the spring of 2014.In total, we solicited 57 active1 master's departments and received a response from 45 individuals identified from 43 different departments.Similarly, 170 individuals, who identified themselves as being from 149 Ph.D.-granting departments (amongst 185 solicited departments) responded to the doctoral survey in the fall of 2013.Overall, the response rate for both samples was above 75% indicating substantial representation of both types of the department.

VI. DATA ANALYSIS
Amongst several other factors, we gathered data on the number of applications received annually (question 2) and each program's intended annual cohort size (question 4).The ratio of applications received to intended annual cohort size is a rough indicator of how many applications the admissions committee reviews for each open slot in the annual graduate cohort-characterized as an admissions pressure that the department faces.
Of interest to the current analysis, we obtained respondents' ranking of the importance of 20 separate admission criteria, each rated independently on a 7-point scale running from "least important" to "most important" (question 12).The weighting of responses was used to balance the impact of each department's contribution; e.g., if there were two respondents from the same department, each contribution was weighted as 0.5 in the calculation of the mean.The responses for each criterion were compared using Wilcox pairwise rank sum tests to evaluate significant differences in the relative importance placed upon various criteria.Further, we compared the weighted mean responses between the master's department data to that of the Ph.D. department data to understand if the similarities or differences in the importance of criteria are significant across the two types of graduate programs.
We also calculated effect sizes for all criteria that were found to be statistically significantly different in the comparison of weighted mean responses.We used Cliff's delta (d) as the estimate for effect size, which is the appropriate measure for ordinal data [25,26].Effect sizes can be categorized [25] as "small" for delta between 0.1 and 0.3, "medium" between 0.3 and 0.5, and "large" between 0.5 and 0.7.
Furthermore, we subsequently selected a subset of both data sets to represent similar "sized" and "pressured" departments (i.e., departments in each sample that reported their annual cohort size was no greater than 10 and that their applications-to-cohort-size ratio was no more than 5) to study if the importance of these criteria continues to persist in the subselection.This subselection was performed because of the wider distribution of cohort size and application pressure faced by doctoral programs, which could confound any differences seen in the overall samples on the admission criteria ranking (e.g., perhaps the larger Ph.D. programs respond differently to certain criteria simply because of their size, not due to other "strategic" reasons).All elements of the analysis reported here were conducted in the statistical software system R [27].

A. Admission pressure
Figure 1 indicates the application pressure-the ratio of applications received to intended annual cohort size plotted against the intended annual cohort size (i.e., capacity to enroll) for master's and Ph.D. programs.
The average size of the master's and Ph.D. programs is 8.6 and 14.2, respectively.As shown in Fig. 1, and following the average size of the master's and Ph.D. programs, the intended annual cohort size for master's programs is typically smaller compared to the Ph.D. programs.Also, Ph.D. programs have a wider distribution of program size in comparison to the master's programs.Master's programs are not only smaller in terms of cohort or enrollment capacity, but also experience less application pressure in comparison to many (smaller as well as large) Ph.D. programs.For instance, the highest admission pressure faced by a Ph.D. program in our data is approximately 180 applications for just 5 positions (for a ratio of 36).Some popular, large, highly rated Ph.D. programs receive upwards of 500 applications.In the case of the master's program, the highest pressure reportedly faced is 20 applications for 2 positions (a ratio of 10).The average application pressure for master's is 2.3 and for the Ph.D. programs is 10.0, respectively.

B. Ranking of admission criteria or students' factors
Next, we analyzed the importance of 20 admission criteria, each rated independently on a 7-point anchored scale for the entire available samples of master's and Ph.D.-granting departments.One item, "u.Other," is not graphed or analyzed here.In addition to ranking the importance of each item, faculty could also choose to indicate that they "don't use" each item.The weighted mean response and standard error for 20 admission criteria for master's and Ph.D. physics programs are graphed in Fig. 2 (the criteria are sorted from highest to lowest mean ranking observed in the Ph.D. programs). 2The departments which specified that they don't use a particular item were eliminated in the mean response and standard error calculation for that item.The top six most important criteria amongst master's programs were as follows: • GPA or grades-physics or math (mean response As seen in Fig. 2, these six criteria received a shared high importance for both master's and doctoral programs (e.g., the mean responses were above the midpoint of the scale for both samples), but in a direct comparison between types of programs, some differences were observed (discussed in Sec.VI C).The lowest rated criteria as reported by master's programs were as follows: • We also analyzed the rates at which each item was indicated as not being used at all.The fraction of master's and Ph.D. programs reportedly not using each of these admission criteria are summarized in Fig. 3.
In comparison to Ph.D. programs, master's-granting departments appear to be using fewer criteria in admissions as the fraction of such programs reporting that they don't use each criterion is higher in virtually every case, perhaps reflecting the lower application pressure they feel.The Ph.D. departments, on the other hand, tend to use a greater number of criteria simultaneously.

C. The relative importance of criteria between master's and Ph.D. program
We divided the 20 admission criterion as "high rated" or "low rated" by comparing its weighted mean response with the midpoint of the scale.The high rated criteria have weighted mean response ≥4.Amongst the 20 criteria, 15 were thusly categorized as "high rated."We then compared the weighted mean response of each master's criteria with that of the Ph.D. program sample and categorized the 20 criteria on the basis of significant (or nonsignificant) differences in the mean. 3We also calculated effect sizes for all the criteria that were identified as significantly different in the comparison of mean responses.Figure 4 shows the four categories developed with this two-way sorting.The horizontal axis divides criteria as high rated (items in category I and II) or low rated (items in category III and IV).The vertical axis divides the criteria as to whether the difference between master's and Ph.D. program responses were significant (items in category II and III) or nonsignificant (items in category I and IV).The effect sizes ranged from 0.24 (small) to 0.46 (medium).We note that in the current analysis the sample sizes are relatively small and that the populations being compared are relatively homogeneous (e.g., both samples represent graduate faculty connected to graduate admissions in physics, only differing by the highest graduate degree offered at the institution) so we would not particularly expect effect sizes to be large or above; however, knowledge of the effect size of the difference between samples is useful to assess the relative, practical importance of various results.In the results below, the differences generally fall into small or medium categories (and large in one case).
Shared, highly rated criteria for both master's and Ph.D. programs appear in category I.The presence of undergraduate courses taken, physics and math GPA, and general GPA in this category is not surprising.Another criterion, undergraduate courses taken, also received shared importance.This result may indicate there are limited opportunities amongst both types of program for students from smaller undergraduate departments that do not offer all necessary canonical course taking.
Amongst the high rated and but significantly different responses (items in category II), each criterion appearing had relatively lower importance in the master's program data than Ph.D. program data.Therefore, the increase in prioritization was stronger amongst Ph.D. programs.Note that both GRE quantitative and GRE physics subject scores appear in this category.That is, both types of institution rated these highly, but Ph.D. programs rated them as statistically significantly higher.A small (with d ¼ 0.24) and medium (with d ¼ 0.42) effect size were observed in the difference between the two programs in terms of GRE quantitative ranking and GRE physics subject criteria ranking respectively.Further, as shown in Fig. 3, a considerable fraction of master's departments do not consider the GRE physics test and GRE quantitative score at all.Therefore, for those master's departments which do consider the quantitative and/or physics GREs in their admissions, these criteria are relatively less important than in the Ph.D. admissions.This is important for the undergraduates who may be considering applying to master's programs but may have the perception that a high GRE quantitative and/or GRE physics score is uniformly necessary across all programs.
Furthermore, amongst the high rated criteria in category II, departmental research opportunities and prior research experience criteria were found to be significantly different between the type of program.A medium effect size (d ¼ 0.31) was observed between the two programs for the departmental research opportunities ranking, and a medium effect size (d ¼ 0.46) was observed for the prior research experience ranking.It may be that faculty in master's programs are less specific about strong research exposure at the undergraduate level as a prerequisite for graduate study.Amongst category III those items rated low but having a significant difference between the types of programs, the mean response of each indicated criterion is lower in the master's data than in the Ph.D. data.For example, the low rated criteria of prior research publications and conference presentations were rated as significantly less important in master's programs.The effect sizes between the two programs for prior research publications and prior conference presentations criteria were medium (d ¼ 0.39) and small (d ¼ 0.29), respectively.As mentioned earlier, we would not particularly expect the effect size to be large or above for our homogenous sample.

D. Comparison of similar sized and pressured
master's and Ph.D. programs In the previous analysis of the importance of different admission criteria (Sec.VI C), we compared the entire available samples of master's programs and Ph.D. programs.This provides a broader picture of the differences between the admission practices of these two populations.However, considering the variations in application pressure discussed in Sec.VI A, one may wonder if the differences seen are simply due to the systematic differences in size and application pressure.Hence, we compared similarly sized and pressured master's and Ph.D. programs to account for such hypotheses.This provides an additional lens to understand if the different types of departments approach admissions differently.Figure 5 below shows the distribution of the subset of all available similarly sized (n ≤ 10) and similarly pressured (applications received to intended annual cohort size ratio ≤5) programs.As can be seen, these subsets are much more comparable in these two factors and facilitate an "apples-to-apples" comparison.This subset includes 33 master's and 28 Ph.D. departments that satisfy the two conditions, representing 73% of the sample of master's departments (a large chunk of this sample), though only 19% of responding Ph.D. departments.Figure 6 shows a graph of the weighted mean response of each admission criteria for the subset of master's and Ph.D. programs.
Figure 7 shows the distribution of admission criteria for the subset according to the same four categories described in Sec.VI C. The criteria "moved" in another quadrant in comparison to their prior position in the catergorization of criteria for all sample (as shown in Fig. 4) are italicized.
In Fig. 7, a total of 13 out of the 15 previously high rated criteria (category I and II in Fig. 4) remains highly rated.GRE verbal scores (mean response 3.90 AE 0.32) and undergraduate institution type (mean response 3.73 AE 0.27) become low rated criteria along with all of the previously reported factors.
There are four factors that were found to be both rated high and significantly different4 between subsamples.These are student research interests, quantitative GRE scores, prior research experience, and departmental research opportunities (all of these were previously found to be statistically different in the complete sample of programs).The effect sizes between the two programs were medium (d ¼ 0.39), medium (d ¼ 0.39), medium (d ¼ 0.32), and large (d ¼ 0.50) for the students research interests, quantitative GRE scores, prior research experience, and departmental research opportunities criterion, respectively.This significant difference indicates that reliance on quantitative GRE scores is persistent even for smaller and low pressured master's and Ph.D. programs.Thus, GRE quantitative scores remain as important admission criteria in many graduate programs.However, smaller and low pressured master's departments assign lesser importance to it in comparison to similar Ph.D. programs, an effect observed previously in the complete sample.A similar result is observed for prior research experience, indicating that smaller, low pressured master's departments value prior research experience but may be more lenient in using it as an admission criterion in comparison to Ph.D. programs.There was no statistical difference in the lower rated criteria in the subsamples (e.g., category III is now empty).

VII. DISCUSSIONS AND CONCLUSION
This study details how different admission criteria are rated in determining admissions to master's programs, and how admission processes are similar and different between master's and doctoral programs.An understanding of these issues is important from a student perspective (who can use this information to make more informed choices about their graduate school) as well as for departments (who can improve their processes and make better decisions in their recruitment and admissions).

A. Research question 1
In considering admissions pressure, a lower ratio of applicants-to-cohort-size is seen in many master's program by comparison to Ph.D. programs.In fact, except for two programs, all the master's departments in our sample report an application pressure of no greater than 5. Similarly, the distribution of cohort size was smaller in comparison to the Ph.D. programs, suggesting faculty had to review relatively fewer applications which may make the admissions decisions process less burdensome.
It is possible that the use of fewer admission criteria overall in master's programs (shown in Fig. 3) reflects the decreased admission pressure experienced by these programs.Nevertheless, several admission criteria were given similarly high importance amongst both master's and Ph.D. programs, indicating a certain consensus on their importance in graduate education irrespective of the type of program; GPA-general, GPA-math and physics, and undergraduate courses taken being three such criteria.

B. Research question 2 and 3
The similar, high importance assigned to the criteria GPA-general, GPA-math and physics, and undergraduate courses amongst both master's and Ph.D. programs is not surprising.Grades and GPA may be interpreted as a reflection of mastery in that subject, and faculty might prefer students for graduate programs who have already exhibited high performance in their presumably foundational undergraduate course taking.The high ranking of undergraduate courses taken also raises a concern for students at those undergraduate institutions which may not necessarily be able to offer all canonical courses in physics (a large fraction-42%of undergraduate degrees in physics are awarded at smaller institutions [28]).So, undergraduate students who do not have the opportunity to take certain canonical courses may miss significant graduate opportunities.For students, this ranking strongly suggests that aspiring graduates need to be proactive during their undergraduate studies to ensure they select particular undergraduate courses (if they are available).
Note also that the emphasis on the canonical undergraduate course taking limits the chances of students from other disciplines to consider physics graduate school options (say, engineering majors who are interested to move into physics research areas).
Regarding the further results on the importance placed on other admissions criteria, GRE quantitative scores were another high rated admission criterion amongst both master's and Ph.D. programs, though in the direct comparison master's programs weigh it relatively lower.So, students with lower quantitative GRE scores may have a somewhat better likelihood of success in admissions by applying to master's programs.Further, in the focused comparison of similar sized and similar pressured master's and Ph.D. programs, the importance placed on the GRE persisted.This suggests that the GRE quantitative test remains important in many master's and Ph.D. admissions irrespective of the type of program (master's or Ph.D.), size, or the application pressure felt by the program.Thus, even smaller or low pressured master's departments are at risk of losing many potential graduate students who may have great potential for research but have a lower GRE quantitative score.Even more problematic, the use of GREs in admissions has been reported to significantly limit the opportunities for students from underrepresented groups [4] and thus may be limiting the diversity of the master's student population.
Letters of recommendations was another high-rated admission criterion amongst both master's and Ph.D. programs.Though it is rated relatively lower in the Master's programs, in the focused comparison of similar sized and similarly pressured master's and Ph.D. programs, the difference becomes nonsignificant.This suggests, in general, all faculty, and particularly those at smaller master's departments highly value recommendation letters in the admissions process.Given that the letters are rated important along with the GPA, GRE scores, and undergraduate courses taken, this may reflect the fact that faculty are interested in gaining a perspective beyond simple GPA, GRE scores, and prior course taking in order to understand their applicants better.The importance placed on letters represents an opportunity for students who may have missed certain courses in their undergraduate programs or may be otherwise limited in their chances of being accepted to a graduate program.Prospective applicants, therefore, may be well served to ways to interact with the letter writers and discuss their strengths, experiences, and interests in graduate research.Further, letter writers should convey applicants' research abilities, strengths and address weaknesses or limitations appropriately.Note that letters receive higher importance in admissions than students' personal statements.So, students should be made aware of why letter writers should know them better.
On the other hand, the whole process of requesting letters, interacting with letter writers, and the letters themselves do raise the risks associated to factors such as implicit bias, and may have a significant impact on diversity [29].Letter writers and readers should be aware of implicit bias, be trained to understand how to deal with it, and should write and interpret recommendation letters with a great care.
In terms of research-related admission criteria, prior research publications and conference presentations were rated relatively low amongst both master's and Ph.D. programs.Furthermore, prior research experience was rated as important but, statistically significantly lower in master's admissions compared to doctoral admissions.This suggests an opportunity for those undergraduates who are not confident about their research preparation, and so may consider applying for master's programs as a path to enhancing their research experience.Likewise, this may also benefit undergraduates who have not fixed their interests in a specific research area.Related to this, about 67% of physics exiting master's programs offer a specific research focus during the master's degree [17].Therefore, master's programs may be more open to accommodate undergraduates without established research interests and support them towards specific post-graduate careers.

C. Limitations of this study
One limitation of this study is that although the survey response rate was above 75%, in most of the cases the responses represent a single individual from each institution.We cannot, therefore, address the variation amongst faculty in their perspectives on admissions practices within departments.Therefore, our data were treated as samples rather than whole population data (despite having a large representation in the population), and we supported our results with statistical inference.Further, the survey data for master's and Ph.D. programs were collected at different times (spring of 2014 and fall of 2013, respectively), although comparison of data samples collected with a significant time interval are not particularly unique.

FIG. 1 .
FIG. 1. Application pressure (ratio of application received to intended cohort size) versus the size of the program.