Gender bias in first-year multiple-choice physics examinations

The multiple-choice section of the final examination for the first-year Advanced Physics I course at the University of Newcastle, Australia between 2010 and 2018 was investigated for gender bias. A Mantel-Haenszel analysis revealed that approximately 20% of the multiple-choice questions exhibited statistically significant gender bias. A schema for characterizing the multiple-choice questions was proposed and used to analyze the entire question set. Male bias questions showed moderate to large bias and tended to include characteristics related to visualization, though not images. Several questions exhibited a moderate bias in favor of females and were characterized by requiring a numerical calculation involving a simple one-step equation. These results indicate that with continued development, gender bias analysis of physics questions based on a characterization schema may be used as a routine tool for testing for the presence and origin of gender gaps in student performance.

Figure 1

Plot of the log transformed Mantel-Haenszel odds ratio (${\alpha }_{\mathrm{MH}}^{*}$) parameter for each MCQ analyzed. The plot is split up into topics and shows gender relationships across these categories. A positive (negative) value of ${\alpha }_{\mathrm{MH}}^{*}$ indicates a bias toward females (males), with a greater probability of a female (male) student answering the question correctly than a male (female) student of the same ability. The gray shaded bars show ${\alpha }_{\mathrm{MH}}^{*}$ for which $p_{\mathrm{MH}}<0.05$. The black shaded bars show ${\alpha }_{\mathrm{MH}}^{*}$ for which $p_{\mathrm{MH}}>0.05$.

Figure 2

The image attached to question four.

Figure 3

The number of times each characteristic is observed within the set of questions used in the first-year advanced physics exam.

Figure 4

Comparison of the percentage distribution of characteristics for male and female biased questions. Out of the eight questions that show gender bias, five are male biased and three are female biased. For example, for the numbers category ($N$), the bars show that 40% of the male bias questions are category $N$ and 100% of the female bias questions are category $N$.