Colorado Upper-Division Electrostatics diagnostic: A conceptual assessment for the junior level

As part of an effort to systematically improve our junior-level E&M I course, we have developed a tool to assess student conceptual learning of electrostatics at the upper division. Together with a group of physics faculty, we established a list of learning goals for the course that, with results from student observations and interviews, served as a guide in creating the Colorado Upper-Division Electrostatics (CUE) assessment. The result is a 17-question open-ended post-test diagnostic (with an optional 7-question pretest) and an accompanying grading rubric. We present measures of the validation and reliability of the instrument and grading rubric, plus results from 535 students in both standard and interactive-engagement courses across seven institutions as a baseline for the instrument. Overall, we find that the CUE is a valid and reliable measure, and the data herein are intended to be of use to researchers and faculty interested in using the CUE to measure student learning.

Figure 1

Abbreviated course-scale learning goals for PHYS301, developed by the faculty working group. The full set of learning goals is available at 20.

Figure 2

Two problems from the CUE, Q3 (5 points) and Q9 (10 points), and their accompanying grading rubrics.

Figure 3

Absolute value grader differences in “total CUE score” for a total of 47 exams graded in common by independent graders using a common rubric. Categories are exclusive of the lowest number in the bin except for the lowest bin: E.g., 0–2.5 can be read as $0\le X\le 2.5$ and 2.5 to 5 can be read as $2.5<X\le 5$.

Figure 4

Inter-rater differences on CUE questions for 52 exams. “Close agreement” is agreement within $\pm 20\%$ ($\pm 1$ point on a 5 point question), “moderate” is within $\pm 20\%–50\%$ ($\pm 1–2.5$ points on a 5 point question), and “poor” is off by 50%–100%. Blanks omitted (thus, exam $N$ varies slightly per question, with a maximum of 52 exams per question).

Figure 5

Histogram of student scores on the CUE based on $N=103$ students in three courses.

Figure 6

Median, mean, and mastery performances on each question on the final version of the CUE for $N=103$ students. “Mastery” level represents the upper 70th percentile on each question; that is, 30% of students scored at or above this level, and 70% scored below.

Figure 7

“Common” CUE scores across institutions for $N=488$ students. “Post-test” represents course average score (% correct) for the subset of CUE questions given in common across all exams (88 out of 118 possible points). “Gain” represents the course average (out of 100%) for the difference between the pretest (60 points) and the matched subset of the post-test (i.e., 60 points). PER courses used the research-based materials developed at CU. STND courses used a standard lecture-based format. Because of the lack of pretests for CU-RES1 and CU STND1 and STND2, pretest scores are estimated (and thus gain scores are effective and lack error bars) based on the stable pretest scores for other semesters. Courses are not listed chronologically. Courses A and B (from final CUE administration) are labeled. Error bars represent 1 standard error of the mean.