Assessing students ’ conceptual knowledge of electricity and magnetism

We present the Electricity and Magnetism Conceptual Assessment (EMCA), a new assessment aligned with second-semester introductory physics courses. Topics covered include electrostatics, electric fields, circuits, magnetism, and induction. We have two motives for writing a new assessment. First, we find other assessments such as the Brief Electricity and Magnetism Assessment and the Conceptual Survey on Electricity and Magnetism not well aligned with the topics and content depth of our courses. We want to test introductory physics content at a level appropriate for our students. Second, we want the assessment to yield scores and gains comparable to the widely used Force Concept Inventory (FCI). After five testing and revision cycles, the assessment was finalized in early 2015 and is available online. We present performance results for a cohort of 225 students at Siena College who were enrolled in our algebraand calculus-based physics courses during the spring 2015 and 2016 semesters. We provide pretest, post-test, and gain analyses, as well as individual question and whole test statistics to quantify difficulty and reliability. In addition, we compare EMCA and FCI scores and gains, and we find that students’ FCI scores are strongly correlated with their performance on the EMCA. Finally, the assessment was piloted in an algebra-based physics course at George Washington University (GWU). We present performance results for a cohort of 130 GWU students and we find that their EMCA scores are comparable to the scores of students in our calculus-based physics course.


I. BACKGROUND
Siena College is a liberal arts college in upstate New York with an enrollment of approximately 3000 students.The Department of Physics and Astronomy at Siena College consists of eight faculty members and approximately 90 physics majors.The department currently offers two versions of the introductory physics sequence.The first is a calculus-based course for freshman physics majors as well as math, computer science, and chemistry majors.The second is an algebra-based course for biology and biochemistry students, and many of these students pursue careers in medicine or the heath care industry.The course for the biology students uses many examples of the physics of the human body.The algebrabased course covers more topics, and thus the time per topic is less in the algebra-based courses.
Six years ago, we began systematically implementing assessments in all introductory physics courses at the beginning and end of each semester to quantify student learning gains.During the first semester, we administer the Force Concept Inventory (FCI) [1], the Lawson Classroom Test for Scientific Reasoning (CTSR) [2], and the Colorado Learning Attitudes about Science Survey (CLASS) [3].For the second semester, we considered using the Brief Electricity and Magnetism Assessment (BEMA) [4], the Conceptual Survey on Electricity and Magnetism (CSEM) [5], and the Determining and Interpreting Resistive Electric Circuit Concepts Test (DIRECT) [6].However, we found these assessments to be overly long and difficult or we found they were not well aligned with the topics and content depth of our courses.We considered creating a shortened assessment based on the available assessments, but we decided instead to create a new assessment, which we call the Electricity and Magnetism Conceptual Assessment (EMCA).
The development of the EMCA began in 2010.Originally a topic list was developed based on the course syllabus and the topics covered in other assessments.The assessments that were reviewed include the BEMA, the CSEM, and the DIRECT.We found the BEMA and the CSEM to be overly difficult and the DIRECT only covered the topic of circuits.We also found the BEMA to have an unusually long list of answer choices, and we were concerned that our students would not read through all of the answers or attempt the assessment with their full effort.
One of the authors drafted questions that addressed the topics and modified them several times until they were in a form to share with the rest of the authors.There were 46 questions in the first assessment.The four authors reduced the assessment to 30 multiple-choice questions with an

II. SAMPLE
For this study, we report on the results of the EMCA assessment for students taking the second semester of our general physics sequence.Data are reported for 225 students with a matched pretest and post-test assessment score.The results represent 115 students from the spring of 2015 and 110 students from the spring of 2016.The results are given for 105 students taking our calculus-based physics course and 120 students taking our algebra-based physics course.
In addition, we report on the results of the correlation of the EMCA and the FCI for a set of 155 matched students who took both assessments.
The EMCA was also piloted at George Washington University (GWU), and we report results for 130 GWU students who took the assessment in the fall of 2015 and spring of 2016.

III. ABOUT THE COURSES
The general physics sequence at Siena College is a fourcredit course in the fall and spring semesters including 3 hours of class time and 3 hours of lab each week.Class sections and lab sections are not linked.Lab material is matched to classroom material, and labs are a mix between inquiry-based labs, data collection and analysis labs, and other activities.The inquiry-based activities are the first exposure to the content while other lab activities extend the knowledge that the students learn from reading and inclass problem solving and other activities.Tutoring is provided by more advanced physics majors five evenings a week for both calculus-and algebra-based courses.
The classroom and lab for the calculus-and algebrabased courses are designed for up to 24 students sitting at rectangular or circular tables, each of which accommodates up to six students.Two computers are located at each table for students to do computer-based activities or data collection and analysis activities.The room has MacBook Air laptops and iMac desktops available for students to research topics and operate lab equipment and simulations.Projectors to screens on opposite walls allow visual access from any seat.

A. Siena College algebra-based physics course
Objectives in the algebra-based physics course address the broad range of topics in introductory physics with an emphasis on topics most relevant to the life sciences and preprofessional health sciences programs and the contents of the Medical College Admission Test.The second-semester course includes three units including electricity and magnetism, optics, and modern topics specific to the health sciences.Approximately one-third of the course time is spent on electricity and magnetism topics.The textbook is Physics: Principles with Applications, 7th ed., by Giancoli [8].Students are expected to read the relevant material and complete homework problems before coming to class.The course is taught in a traditional lecture style interspersed with students completing preassigned problems on the board during class.The course includes weekly quizzes focused on conceptual questions, while two exams include mathematical problems, and no final exam is given.Three instructors taught the courses during the spring of 2015 and the spring of 2016 when the EMCA assessment was given.

B. Siena College calculus-based physics course
The second-semester physics course focuses on the simple harmonic oscillator and waves, optics, and electricity and magnetism.Two-thirds of the course time is spent on electricity and magnetism topics.The textbook is Physics for Scientists and Engineers, 3rd ed., by Knight [9], and students are expected to read assigned chapter sections before coming to class.Students submit solutions to problem sets assigned for homework along with online MasteringPhysics tutorials.Class time is a mix of short lectures frequently interspersed with think-pair-share questions and other mini problems, on content in which known misconceptions exist or on content that students typically find difficult.The main role of the instructor during class time is to facilitate students working through problems and in-class activities.Each week or two, 30-minute quizzes are given on one or two chapters, and the questions are a mix of multiple choice, short-answer conceptual questions, and one or two longer problems.The students are also given a midterm and a comprehensive final exam.Three instructors taught the courses during the spring of 2015 and the spring of 2016 when the EMCA assessment was given.These instructors were not the same as the ones teaching the algebra-based courses.

C. George Washington University algebra-based physics course
The algebra-based physics course at George Washington University is a four-credit course including a lab.The course is taught in both the fall and the spring semesters.Approximately two-thirds of the semester is spent on electricity and magnetism topics, and the remaining time is devoted to modern physics topics.The SCALE-UP model of teaching physics is used, and the enrollment in each course is 40-50 students.Students are arranged in round tables that accommodate 6-9 students.The method is similar to the approach in the Siena calculus-based course.The textbook is College Physics, by Giambattista et al. [10].One-half to two-thirds of students enrolled in the course are premed students with the remaining students comprising a mixture of liberal arts majors.Two instructors taught the courses during the fall of 2016 and the spring of 2016 when the EMCA assessment was given.

IV. INDIVIDUAL QUESTION INFORMATION AND STATISTICAL ANALYSIS
The topics and corresponding question numbers included on this test are shown in Table I.In Figures 1-3, we show three sample questions from the assessment.Questions 4, 21, and 1 are examples of some of the least, medium, and most difficult questions, respectively.Item difficulty is addressed in more detail in the next section.Post-test averages of item difficulty, discrimination, and point-biserial coefficient for  II.The full test is available online [7].
The student responses on the pretest and post-test for the algebra-based course and calculus-based course are shown in Figs. 4 and 5, respectively.The black histogram shows the normalized distribution of choices for the pretest, the red histogram shows the normalized distribution of choices for the post-test, and the cyan vertical line represents the correct answer.For example, on the pretest, students selected all possible answer choices for question 13, with answer (C) selected most often.On the post-test, all answers were selected by the algebra students with the incorrect answer (C) still being selected more often than the correct answer (D).However, for the calculus students, only three choices were selected and the correct answer (D) was selected most frequently.The histograms of student choices overlaid with correct answers in Figs. 4  and 5 provide a graphical visualization that matches the item difficulty values for each question as described in the next section.All distractors for each question were chosen by students on the pretest while some distractors were not chosen for a few questions on the post-test.
Students in both courses found question 15 difficult.The question tests students' knowledge about parallel circuits and the relationship between resistance, current, and voltage.The difficulty is not surprising as students are often confused by the voltage drop for series and parallel circuits.We used the same statistical measures described by Ref. [4] for the BEMA for the individual question and the whole test analysis for this assessment.For simplicity, we use the same representation and equations that were used in Ding's analysis.The statistical analysis for individual questions is given in Table II.Item difficulty, item discrimination, and point-biserial coefficient is given for each question.The results in Table II are based on post-test data from two academic years when the assessment was in its final form.The analysis includes all students that took the assessment in the spring of 2015 and the spring of 2016 along with results for the two different types of courses.

A. Item difficulty
Item difficulty measures the fraction of students that answer the question correctly.The item difficulty is calculated as the number of correct answers divided by the total number of students who took the test: where P is the item difficulty index, N C is the number of students that selected the correct answer, and N is the total number of students.Note that a higher value for item difficulty indicates an easier question, while a lower value reflects a harder question.The ideal value for the item difficulty index is 50% [11].
In this assessment, the hardest question is number 25 with an item difficulty of 0.171, the easiest question is number 4 with an item difficulty of 0.736, and the average and standard deviation for all questions is 0.454 AE 0.137.Comparing this to our results for the FCI for the same students, the hardest question has an item difficulty of 0.175 and the easiest question has an item difficulty of 0.799, with an average and standard deviation for all questions of 0.517 AE 0.172.We chose a widely adopted criterion that requires the difficulty index value to be between 0.3 and 0.9 with an ideal value of 0.5 (see, e.g., Refs.[4,5,12]).Our average EMCA value is very close to the ideal value of 0.5.Three questions are below but close to the lower limit of 0.3.Our hardest question has an item difficulty of 0.171, but has been reviewed and the decision was made to keep it.For comparison, the CSEM has a range of item difficulty between 0.1 and a little over 0.8 [5], and the BEMA has a range of item difficulty of slightly below 0.2 to slightly above 0.8 [4].

B. Item discrimination
Item discrimination refers to the ability of an item to differentiate among students on the basis of how well they know the material being tested [12].Students with strong content knowledge typically answer questions correctly while students with weak content knowledge answer questions incorrectly.A high item discrimination index reflects that the question is able to distinguish between strong and weak students.
For item discrimination, we chose to split the scores between high and low with 50% in the high scoring group and 50% in the low scoring group to calculate the item discrimination indices.The item discrimination index is given by where D is the item discrimination, N H is the number of students that selected the correct answer from the high scoring group, N L is the number of students that selected the correct answer from the low scoring group, and N is the total number of students.
In this calculation, the number of samples is arranged by test score and split in the middle with a high scoring group represented by N H and a low scoring group represented by N L .Theoretically, the range for the discrimination index is ½−1; 1.One commonly used criteria for the discrimination index value for each item is D ≥ 0.3.Another suggested cutoff value is D ≥ 0.2 [12].Doran also notes that the item discrimination is strongly influenced by the item difficulty.In this assessment, the question with the lowest discrimination index is question 25 with a value of 0.081, while all other questions have indices above 0.13.Questions 10 and 21 have the highest discrimination index with a value of 0.520.The average item discrimination is 0.343ðAE0.104Þ,which is above the desired value of D ≥ 0.3 [12].This is similar to the values used for the BEMA [4] and the CSEM [5].While question 25 is very difficult and has a low discrimination index, we reviewed the question many times and decided to keep it in the assessment.The question asks about the potential difference between two ends of a metal rod moving in a magnetic field.This is a complex question that requires the students to realize the charge is free to move in a conductor, a moving charge experiences a force that causes the charges to gather at one end of the rod, and finally, the student needs to connect voltage to a separation of charge.

C. Point-biserial coefficient
The point-biserial coefficient indicates the correlation between a student's individual question response to the student's score on the test [13].If the coefficient is high, the item is positively correlated with the test, and if the coefficient is low, the item is negatively correlated with the test.
The point-biserial coefficient r pbc is calculated with where X1 is the average total score of students who correctly answered the question of interest, X is the average total score for the entire sample, σ is the standard deviation of the total score for the full sample, and P is the item difficulty for the question of interest.For this test, pointbiserial coefficients range from 0.233 to 0.600, with an average of 0.454ðAE0.095Þ.The criterion widely adopted for measuring the consistency of a test question is r pbc ≥ 0.2 [14].The point-biserial coefficient is above 0.2 for each question for the combined results of algebraand calculus-based courses.No questions fall below this value for the students in the algebra-based courses.Two questions fall below this value for the students in the calculus-based courses.However, the point-biserial coefficients for those two questions were 0.191 and 0.193, very close to the criterion.

V. ENTIRE ASSESSMENT INFORMATION AND STATISTICAL ANALYSIS A. Test results
Total assessment results are given in Table III.The data in this table include the pretest score, post-test score, and normalized gain.Normalized gain is a common metric used to measure student gains and is defined as the ratio of the number of points gained between the pre-and post-test divided by the maximum possible gain [15]: The normalized gain calculated for Table III is the average of the each student's normalized gain value.Negative values for normalized gain represent students who scored lower on the post-test than the pretest and these negative values were included in the calculation.
A total of 225 students with matched pre-and post-test assessment scores (115 students for the spring of 2015, 110 Up until this point, the analysis has focused on individual questions.The two statistical functions used for a fulltest analysis include the Kuder-Richardson (KR) reliability index [16] and Ferguson's delta [17].A summary of the statistical analysis for the test as a whole calculated from post-test data is given in Table IV.

B. Kuder-Richardson reliability index
The Kuder-Richardson reliability index measures the self-consistency of the entire test [16].Typically, consistency is measured by having students take the same test multiple times and comparing results.However, the KR method is an equivalent technique that uses a subset of questions from the test and compares them against another subset of the test to check for consistency.This particular analysis (kr21) is specifically for multiple-choice tests and uses each question as a parallel test to check for consistency.The index is given by where R kr21 is the index, K is the number of test items, P is the item difficulty, and σ is the standard deviation of the total test score.The possible range is [0, 1].The acceptable criterion of the kr21 index ≥ 0.7 was used for this test [18].For reference, the BEMA kr21 index is 0.85 [4] and the CSEM kr21 index is 0.75 [5].We calculated our kr21 index for the 2015 and 2016 courses, and the result is shown in Table IV.The indices for the calculus-and algebra-based courses are 0.84 and 0.86, respectively.Therefore, the criterion was achieved for both courses.

C. Ferguson's delta
Ferguson's delta measures the discriminatory power of the test by looking at the distribution of scores over the range [17].An ideal test will have scores spread over the entire range from a total test score of 0-30 points.The Ferguson's delta is given by where N is the number of students in a sample, K is the number of test items, and f i is the frequency of cases at each score.The range of values for δ is [0, 1].The acceptable criterion is 0.9 for good discrimination, see Ref. [14] and p. 31 of Ref. [19].For reference, the delta for the BEMA is 0.95 [4].The deltas for the calculus-and algebra-based courses for our assessment for the 2015 and 2016 courses are 0.97 and 0.97, respectively.The test meets the criterion for both courses.

VI. COMPARING THE EMCA TO THE FCI FOR THE SAME STUDENT SET
As an additional check for the final version of the assessment, we compare our results for the EMCA to the results of the FCI for the same set of students.In Fig. 6 The distribution of scores indicates that some students do score at the high end of the scale for the EMCA in the calculus-based courses.More students reach the highest ranges on the FCI.A few students in the algebra-based courses score at the high end of the scale for the EMCA.A few more reach the highest ranges on the FCI.
We also ran the statistical analysis on our FCI data for the same set of students as a comparison to our new EMCA assessment.These results are included in Table IV data fell within the standard deviation of the EMCA data.The KR index for the FCI data was higher than the KR index for the EMCA data with both above the desired goal for that index.The Ferguson's delta for the FCI was about the same as the value for the EMCA and was above the goal for that metric.
A comparison of the pretest scores, post-test scores, and gain analysis for students who took both the EMCA and the FCI is given in Fig. 8.A Spearman rank correlation test analysis was run for the three sets of scores.The results of all three show that there is a statistically significant correlation between the EMCA and FCI pretest, post-test, and gain scores indicated by values ρ ¼ 0.26 and p ¼ 1.6e −3 , ρ ¼ 0.33 and p ¼ 3.8e −5 , ρ ¼ 0.55 and p ¼ 3.1e −13 , respectively.
A final comparison of the pretest scores, post-test scores, and gain analysis for women and men for the EMCA is given in Fig. 9. Mean and standard deviation values are shown in Table V.A 2-sample Kolmogorov-Smirnov (KS) test was run for the three sets of scores comparing gender for each and the results are listed in Table VI.We found a mildly statistically significant difference between men and women in the calculus-based courses for the pretest scores  (p value of 0.03), but not for the algebra-based pretest scores.We found that there is no statistically significant difference between the genders for the post-test scores and the normalized gain (p values > 0.05).The 2-sample KS test was used because this test allows for sample distributions of different lengths.
While at first glance this may seem surprising, in 2008 Pollock [20] found a similar result with no statistical significance between genders on the CSEM.Specifically, Pollock found that the gender gap is (statistically significantly) increased on the BEMA, and (insignificantly) decreased on the CSEM.

VII. DIFFERENCES BETWEEN COURSES AND PILOT TESTING AT A LARGE UNIVERSITY
Table III shows that while the pretest scores are similar for Siena College's calculus-and algebra-based physics courses, the Siena algebra-based courses have lower posttest and gain scores than the calculus-based courses.Two factors that could account for these differences are the shortened amount of time students spend on electricity and magnetism topics in our algebra-based courses and the style of teaching.One-third of the course time is spent on electricity and magnetism topics in the algebra-based course as compared to two-thirds of the course time for the calculus-based course.The algebra-based courses at Siena are taught in a traditional lecture-based format, while the calculus-based course is taught in an interactive style with preparation expected before class and group problemsolving and short lab-style activities during class.
We report the pretest, post-test, and gain scores for the GWU students in Table III.The pretest average score is similar for all courses.The post-test and normalized gain average values for the Siena calculus-based course and the GWU algebra-based course are very similar.Both courses spend approximately the same amount of time on electricity and magnetism topics.These results help to convince us that the EMCA is appropriate for small liberal arts colleges and large universities, and for algebra-based and calculusbased courses.

VIII. EMCA VALIDITY
In the spring of 2017, we conducted an additional study to gather information about student responses to the EMCA and how these same students perform on the BEMA.Students recorded their reasoning for their answer choices to the EMCA in audio format and the results were analyzed.The goal of the study is to provide more information to potential adopters of the assessment.Seventy-four students in our calculus-and algebra-based general physics courses and junior-level electromagnetic theory course took the EMCA assessment and the BEMA assessment.In addition, students were asked to answer a Likert-scale question about whether the EMCA assessment reflected the material taught in their courses.
A comparison of scores for the BEMA and the EMCA assessments is plotted in Fig. 10.The solid black line shows a one-to-one correlation between the EMCA and the BEMA.The EMCA scores are higher than the BEMA scores, as illustrated by the fact that the data points lie below the line.A Spearman rank correlation shows that there is a statistically significant correlation between the BEMA and EMCA post-test scores (ρ ¼ 0.66 and p ¼ 1.6e −9 ).The Spearman rank correlations for the EMCA and BEMA reflect that student performance on the EMCA and BEMA are related.As reported above, student performance on the EMCA and FCI are related.Figure 8(b) shows that the EMCA post-test scores are comparable to the FCI post-test scores, meeting one of our objectives.At the end of the EMCA assessment given in 2017, students were asked a Likert-scale question about whether the assessment reflects the topics that were covered in their current courses, calculus-or algebra-based general physics courses or the junior-level electromagnetic theory course.Answer choices range from strongly disagree (1) to strongly agree (5).The average answer for the 74 students is 4.01 out of 5 with a standard error of the mean of 0.01, indicating that the EMCA assessment reflects the topics covered and the assessment has strong face validity.
We selected 22 questions from the EMCA assessment and student answers to these questions were analyzed.The reasons for our selections are listed Table VII.Questions 2, 3, and 5 went through many iterations of changes to the stems and alternatives and the figure (question 5) until the questions discriminated well.Question 27 was added toward the end of the process, and we wanted to learn about students' understanding of magnetic flux.Question 25 is the hardest question, and we wanted to ensure that students that selected the correct answer were using the correct reasoning and not guessing.The remaining 17 questions reflect topics listed in Table I, and the reasons for their selection are given in Table VII.
Questions, distractors, and a summary of student responses is included in the Appendix.The results show that students on the post-test are using correct reasoning when they select the correct answer and most students do not guess the correct answer.A few misconceptions were identified.For example, in question 1, students did not recognize charge induction when a charge is near a metal sphere.For question 16, many students do not understand that removing a resistor from a parallel circuit will not increase the current to the remaining element.Students find questions 25 and 27 to be the most difficult, but those that choose the correct answer are using the appropriate reasoning.

IX. CONCLUSIONS
We developed a new assessment tool for our secondsemester freshman-level introductory physics courses.We find other assessments such as the Brief Electricity and Magnetism Assessment and the Conceptual Survey on Electricity and Magnetism not well aligned with the topics and content depth of our courses.We want to test introductory physics content at a level appropriate for our students.Also, we want the assessment to yield scores and gains comparable to the widely used Force Concept Inventory.We performed a statistical analysis of student data on individual questions as well as the test as a whole with results that met all of the criteria for an assessment instrument.Our EMCA scores are correlated with our student scores for the FCI.Together, the statistical analysis of the assessment along with the comparison to the robust well-tested FCI gives us confidence that this newly developed assessment will better align with the second semester of our calculus-and algebra-based general physics courses.While EMCA scores are correlated with BEMA scores, EMCA scores are higher and comparable with FCI scores.The level of difficulty is appropriate for a freshmen-level course and the topics align well with the course content.

ACKNOWLEDGMENTS
This material is based upon work supported by the National Science Foundation under Grant No. AST-0847430.The authors would like to thank Jerry Feldman and his colleagues at GWU for piloting the assessment and sharing their results.The authors would also like to thank John Moustakas at Siena College for creating our online data collection system for assessments.

APPENDIX: STUDENT EXPLANATIONS OF THEIR ANSWERS
In 2017, students took the post-test EMCA assessment and explained their answers immediately after taking it.The explanations were recorded in audio format.Of the 30 questions in the assessment, 22 were selected for analysis.Reasons for the choice of questions are listed in Table I.Each question includes the stem, distractors, and a summary of student responses.The questions are organized according to the topics they address.Question 1 (Fig. 11): A small positive charge, þq, is brought near but does not make contact with a small metal sphere, as shown in the diagram below.The metal sphere is (A) There is no force between the metal sphere and the þq charge.(B) The metal sphere will be repelled by the þq charge.(C) The metal sphere will be attracted to the + q charge.
For question 1, students selected answers A, B, and C with the majority choosing the incorrect answer, A. Their rationale is that the metal sphere is neutral, and according to Coulomb's law, if the sphere is not charged, the force is zero.Those that selected answer C correctly reasoned that the positive charge attracts the negative charges and repels the positive charges in the sphere, attracting the sphere to the charge.The few that selected the incorrect answer B either guessed or thought the charge touched the sphere, transferring positive charge, and repelling the sphere from the charge.Question 20 (Fig. 12): The diagram below shows the cross sectional view of a neutral metal.Within the metal is an insulated hollow cavity that contains a small charge, þq.Which statement is correct?(A) The charge on the inner and outer surfaces of the metal is 0. (B) The charge on the inner and outer surfaces of the metal is þq.(C) The charge on the inner and outer surfaces of the metal is −q.(D) The charge on the inner surface of the metal is þq and the charge on the outer surface of the metal is −q.(E) The charge on the inner surface of the metal is −q and the charge on the outer surface of the metal is þq.
For question 20, students selected answers A-E with the majority choosing the correct answer, E. They reasoned correctly that the positive charge inside attracts the negative charge to the inside and repels the positive charge to the outside of the conducting shell.Students that incorrectly selected answer D reasoned that the metal shell must be negative to make the net charge zero.Students selecting incorrect answers A, B, and C discussed Gauss's law, guessed, or did not provide a reason.
It is interesting that more students recognized charge induction in question 20 than in question 1.In question 1, students treated both the charge and the metal sphere as point charge objects.Perhaps if the metal sphere is made to be much larger than the charge, students may consider charge induction.
Question 2 (Fig. 13): The diagram below shows two small charges ðq 1 ; q 2 Þ.If q 2 is moved so that its distance from q 1 is doubled, the magnitude of the electric force on q 2 will be For question 2, students selected answers A-E, with the majority choosing the correct answer, E, and explaining their choice using Coulomb's law and the fact that the force is proportional to the product of the charge.When students said they guessed, they did not select E. When students incorrectly selected A or B, that the force was tripled or doubled, respectively, they usually explained their answer with respect to one charge only.One student selected C because they said they did not remember the formula for force.Question 3 (Fig. 14): The diagram below shows two small charges ðq 1 ; q 2 Þ.If q 2 is moved so that its distance from q 1 is doubled, the magnitude of the electric force on q 2 will be  (D) reduced to one-fourth the original value.
(E) reduced to one-half the original value.
For question 3, students selected answers C, E, and D. All students that chose the correct answer, D, explained their answer using Coulomb's law and the fact that the force is proportional to the inverse of the square of the distance between the charges.Students that selected the incorrect answer D remembered that force is proportional to the distance, but forgot the fact that the distance was squared.
Students that selected answer C were not sure of their reasoning for that answer.
Question 5 (Fig. 15): Three small positive charges ðþq; þ2q; þ3qÞ are enclosed by three surfaces ðS 1 ; S 2 ; S 3 Þ, as shown in the diagram below.If the net electric flux through S 1 is ϕ E , the net electric flux through For question 5, students selected answers A, B, C, and E. Most students chose either B or C. Students that picked the correct answer C explained their reasoning correctly.They stated that surface 2 contained 3 charges and since surface 1 had one charge with flux ϕ E , the flux through surface 2 must be 3ϕ E .Students that chose the incorrect answer B explained that if surface 1 enclosed charge q, then surface 2 enclosed charge 2q with a corresponding flux of 2 times ϕ E .Students that picked A either guessed or one student explained that if the area increased by the same amount as the charge, the flux would be the same.Two students picked answer E and explained correctly that S2 encloses 3q and the flux is 3ϕ E , and they were not sure why they picked E.
Question 7 (Fig. 16): In the diagram below, a small negatively charged particle, −q, is released from rest in a region containing a uniform electric field.The electric field, E, is directed downward.The force exerted on the negatively charged particle as it is released from rest in the electric field region is directed (E) There is no force exerted on the negatively charged particle as it is released from rest in the electric field region.
For question 7, students selected answers A-E.Half of the students selected answer A and explained their reasoning in two ways.One group of students said that positive charges follow field lines, so negative charges move against the direction of the field lines.The other group explained that the electric field points from positive to negative charge and the negatively charged particle will be either attracted to the positive or repelled by the negative charge that creates the electric field.A quarter of the students selected the incorrect answer B and explained that the charge follows the field lines.Some did not notice that the charge was negative.Students that incorrectly selected C, D, and E used the right-hand rule, guessed, or did not provide an explanation.
Question 8 (Figs. 17  vectors in the positive x direction remain.Another group explained that the field goes from positive to negative.Students that incorrectly selected answer E thought the field from the charges canceled out at P. Students that selected incorrect answers A and C mistakenly thought that the field points from negative to positive charge or they mistakenly canceled the horizontal components of the field instead of the vertical components.

Use the statement and figure below
for questions 11-13 In the circuit below, the resistance of bulb 1 is greater than the resistance of bulb 2.
Question 11 (Fig. 19): Compare the potential difference across the bulbs.
(A) The potential difference across bulb 1 is greater than the potential difference across bulb 2. (B) The potential difference across bulb 1 is less than the potential difference across bulb 2. (C) The potential difference across bulb 1 is the same as the potential difference across bulb 2. (D) It is not possible to compare the potential difference across the bulbs.
For question 11, students selected answers A-D.Most students correctly selected answer A, identified the series circuit, and used Ohm's law to explain that bulb 1 has a larger resistance and has the largest potential difference.Students that selected the incorrect answer B explained that bulb 1 has a larger resistance and a lower potential.Students that selected the incorrect answer C believed that the potential difference is the same for a series circuit or guessed.The student that selected incorrect answer D guessed.
Question 12 (Fig. 19): Compare the current through the bulbs.
(A) The current through bulb 1 is greater than the current through bulb 2.
(B) The current through bulb 1 is less than the current through bulb 2. (C) The current through bulb 1 is the same as the current through bulb 2. (D) It is not possible to compare the current through the bulbs.
For question 12, students selected answers A-C.Most students correctly selected answer C, knowing that current is the same in a series circuit.Students that selected answer B incorrectly explained that a higher resistance results in a lower current.Students that selected incorrect answer A explained that current goes through bulb 1 first and then bulb 2.
Question 13 (Fig. 19): If bulb 1 is removed from its socket, bulb 2 will For question 13, students selected answers A, C, and D. Most students correctly selected answer D and explained that if one bulb is removed from a series circuit, no current flows to the other bulb.Students that incorrectly selected answer C assumed that the second bulb would be reconnected and explained that the current would increase, increasing the brightness.Students that selected incorrect answer A said they did not know that the bulb was actually removed.

Use the statement and figure below
for questions 14-16 In the circuit below, the resistance of bulb 1 is greater than the resistance of bulb 2.
Question 14 (Fig. 20): Compare the potential difference across the bulbs.For question 14, students selected answers A-D.Three quarters of the students selected the correct answer C and explained that the potential difference is the same for a parallel circuit.Students that selected answer A incorrectly used Ohm's law, guessed, or did not provide a reason.Students that incorrectly selected answer B identified the circuit as a parallel circuit and explained that the potential splits for parallel circuits or guessed.
Question 15 (Fig. 20): Compare the current through the bulbs.
(A) The current through bulb 1 is greater than the current through bulb 2. (B) The current through bulb 1 is less than the current through bulb 2. (C) The current through bulb 1 is the same as the current through bulb 2. (D) It is not possible to compare the current through the bulbs.
For question 15, students selected answers A-C.Most students chose the correct answer B and recognized the parallel circuit, knew that the voltage is the same in a parallel circuit, and explained that bulb 1 has a higher resistance.They used Ohm's law to explain that current is inversely proportional to resistance.Students that selected incorrect answer C recognized the parallel circuit and incorrectly stated that current is constant in a parallel circuit.Students that selected A incorrectly stated that current is proportional to resistance, though most recognized the parallel circuit.
Question 16 (Fig. 20): If bulb 1 is removed from its socket, bulb 2 will (A) maintain approximately the same brightness.For question 16, students selected answers A, B, C, and E. Approximately half of the students selected incorrect answer C and reasoned that all of the current goes through bulb 2 when bulb 1 is removed.Approximately half of the remaining students correctly selected answer A and explained that the voltage and current are the same in bulb 2 regardless of whether bulb 1 is present or not.Students that selected answers B and E guessed.For question 18, students selected answers A-D.70% of students selected the correct answer B. Their explanations varied with some using the formula that the resistance is inversely proportional to the area, a few mentioned that wire 2 is a choke, others used an analogy and compared the wires to hallways and larger hallways allow more students through.Some students who selected the incorrect answer A believed the resistance was proportional to the area.Another student who selected A claimed that a greater volume of the same resistivity will have a higher resistance.The few students that selected incorrect answers C and D guessed or claimed that the problem said the resistors were identical in every way so their resistance must be identical.
Question 19 (Fig. 22): The resistors shown in the diagram below are identical in every way except that the length of resistor 1 is greater than the length of resistor 2 ðl 1 > l 2 Þ.Compare the resistance of the two resistors.
(A) The resistance of resistor 1 is greater than the resistance of resistor 2. (B) The resistance of resistor 1 is less than the resistance of resistor 2. Which arrow best approximates the direction of the magnetic field at point P? (C) The resistance of resistor 1 is equal to the resistance of resistor 2. (D) It is not possible to compare the resistance of the resistors.
For question 19, students selected answers A-D.Most students selected the correct answer A. Their explanations varied with some using the formula that the resistance is proportional to the length.Another explained that voltage drop increases along the wire and as the length of the wire goes to infinity, the resistance goes to infinity.Another used an analogy and compared the resistor to a bumpy road and suggested that driving down a long bumpy road would be more difficult than driving down a short bumpy road.Some students who selected the incorrect answer C assumed that resistance does not depend on length.Students that selected the incorrect answer D guessed.
Question 21 (Figs.23 and 24): Two long wires ðW 1 ; W 2 Þ lying perpendicular to the plane of the paper and seen in cross section are shown in the diagram below.The wires carry equal electric currents.Point P is equidistant from the wires.There is a net magnetic field, B net , due to the two wires.If the magnetic field at point P is directed down, which configuration correctly indicates the direction of the electric currents in the two wires?(E) There is no electric current configuration in the two wires that could produce a net magnetic field that is directed down at point P.
For question 21, students selected answers A-D.Half of the students selected the correct answer D. They correctly explained that they used the right-hand rule with many explaining the direction of their thumb represented the current and their fingers wrapping around and representing the magnetic field.Students that incorrectly selected choices A, B, and C also used the right-hand rule.A few explained that their fingers wrapped around the wire, but the direction of the magnetic field was not tangent to their fingers.A few used the term force instead of magnetic field when explaining the vector at P.  (5) The magnetic field at point P is zero.
For question 28, students selected answers A-E.Most students selected the correct answer B using the same reasoning as described in question 21.Students who selected answers A, C, D, and E used the right-hand rule incorrectly.One student explained that the direction of B is in the same direction as W.
Comparing student responses to questions 21 and 28 on the topic of the magnetic field of a current carrying wire, students performed better on easier question 28 than on 21 in which they only considered the magnetic field of one wire (Table VIII).However, students who answered correctly used the same reasoning and no student guessed the correct answer.Most students that answered incorrectly for both questions used the right-hand rule incorrectly.
Question 22 (Fig. 27): In the diagram below, a small positively charged particle, þq, moves horizontally with velocity, v, into a region containing a uniform magnetic field.The magnetic field, B, is directed downward.The force exerted on the positively charged particle as it initially enters the magnetic field region is directed  For question 25, students selected answers A, B, C, and D. Students that selected the correct answer D explained their reasoning correctly.The force on the electrons in the metal bar is down, causing a separation of charge, resulting in a larger potential at the top of the bar.Incorrect answer B was selected most often.Some students explained that the rod was neutral and there was no charge separation.Others said the bar traveled at the same speed so there was a constant potential along the rod.Many students guessed B as an answer.A few students guessed answers A and C. One student explained the reasoning correctly, and selected C because he switched the direction of the force for a negative charge.For question 27, students selected answers A, C, D, and E. Students that picked the correct answer D explained their reasoning correctly, except for one who guessed and did not provide any reasoning.Except for one guess, the students that selected D reasoned that the net flux must be zero.Some mentioned Gauss's law for magnetic fields.The majority of students selected answer A and their explanations varied from each face must be the same or they guessed.Some students selected C and explained that the net flux of the remaining sides must match the flux through one face.Students who picked E guessed or said they were not sure.

FIG. 1 .
FIG. 1. Question 1, an example of one of the most difficult questions, from the EMCA assessment.

FIG. 4 .
FIG. 4. EMCA question choices and answers for the algebrabased classes.Black represents the pretest selection and red represents the post-test selection.The cyan vertical line marks the correct answer.The y axis shows the fraction of the overall population that selected each choice.The dashed horizontal line marks 50% of the total number of students with matched pre-and post-test results.
, the distributions of EMCA pretest scores (gray histogram) and post-test scores (red histogram) for students in calculus-based courses are shown at the top left.On the top right, the distribution of gains for students in the calculus-based courses is shown in blue.We show the same quantities for the algebra-based students in the bottom row.Students in our calculus-based courses have higher post-test and gain scores on the EMCA.In Fig. 7, the distributions of FCI pretest scores (gray histogram) and post-test scores (red histogram) for students in calculus-based courses are shown at the top left.On the top right, the gains for students in the calculus-based courses is shown in blue.We show the same quantities for the algebra-based students in the bottom row.Students in our calculus-based courses have higher post-test and gain scores on the FCI.
FIG. 6. Top left: Distribution of EMCA pretest scores (gray histogram) and post-test scores (red histogram) for students in calculus-based courses.Top right: Normalized gain values for students in the calculus-based courses.Bottom left: Distribution of EMCA pretest scores (gray histogram) and post-test scores (red histogram) for students in algebra-based courses.Bottom right: Normalized gain values for students in the algebra-based courses.Students in our calculus-based courses have higher posttest and normalized gain values.

FIG. 8 .
FIG. 8. Top: Pretest Scores of EMCA versus FCI.Middle: Posttest scores of EMCA versus FCI.Bottom: Normalized gain values of EMCA versus FCI.A Spearman rank correlation test analysis shows that there is a statistically significant correlation between the two scores for the 2015 and 2016 students.

FIG. 7 .
FIG. 7. Top left: Distribution of FCI pretest scores (gray histogram) and post-test scores (red histogram) for students in calculus-based courses.Top right: Normalized gain values for students in the calculus-based courses.Bottom left: Distribution of FCI pretest scores (gray histogram) and post-test scores (red histogram) for students in algebra-based courses.Bottom right: Normalized gain values for students in the algebra-based courses.As with the EMCA results, students in our calculus-based courses have higher post-test and normalized gain values.
reduced to one-fourth the original value.(E) reduced to one-half the original value.
(A) upward.(B) downward.(C) into the plane of the paper.(D) out of the plane of the paper.

(
A) maintain approximately the same brightness.(B) become approximately half as bright.(C) become approximately twice as bright.(D) not light.(E) It is not possible to determine what will happen to bulb 2 if bulb 1 is removed from its socket.
(B) become approximately half as bright.(C) become approximately twice as bright.(D) not light.(E) It is not possible to determine what will happen to bulb 2 if bulb 1 is removed from its socket.

Question 18 (
Fig. 21): The resistors shown in the diagram below are identical in every way except that the radius of resistor 1 is greater than the radius of resistor 2 ðr 1 > r 2 Þ.Compare the resistance of the two resistors.(A) The resistance of resistor 1 is greater than the resistance of resistor 2. (B) The resistance of resistor 1 is less than the resistance of resistor 2. (C) The resistance of resistor 1 is equal to the resistance of resistor 2. (D) It is not possible to compare the resistance of the resistors.
Question 28 (Figs. 25 and 26): A long wire, W lying perpendicular to the plane of the paper and seen in cross section is shown in the diagram below.The wire carries a constant electric current that is directed out of the plane of the paper.

Question 27 :
The magnetic flux through one face of a cube is þϕ B .The net magnetic flux through the other five faces of the cube is(A) 5ϕ B .(B) −5ϕ B .(C) ϕ B .(D) −ϕ B .(E)It is not possible to determine the net magnetic flux through the other five faces of the cube without knowing the strength of the magnetic field and the dimensions of the cube.

TABLE I .
Conceptual topics and corresponding question numbers, with pre-and post-test results (number of students that answered correctly out of total number of students).

TABLE II .
Post-test individual question item difficulty, discrimination, and point-biserial coefficient.

TABLE III .
Average scores and standard deviation for pretest, post-test, and normalized gain.

TABLE IV .
Post-test statistics of assessment.(Average and standard deviation results are given for the EMCA and FCI assessments for the Siena College calculus-and algebra-based courses).

TABLE V .
Average scores and SD for pretest, post-test, and normalized gain by gender.
FIG.10.BEMA scores versus EMCA scores.A Spearman rank correlation test analysis shows that there is a statistically significant correlation between the BEMA and EMCA scores for the 2017 students.

TABLE VII .
Reasons for selecting questions.

TABLE VIII .
2017 student answer choices.