Effectiveness of Tutorials for Introductory Physics in Argentinean high schools

This longitudinal study reports the results of a replication of Tutorials in Introductory Physics in high schools of a Latin-American country. The main objective of this study was to examine the suitability of Tutorials for local science education reform. Conceptual learning of simple resistive electric circuits was determined by the application of the single-response multiple-choice test “Determining and Interpreting Resistive Electric Circuits Concepts Test” (DIRECT) to high school classes taught with Tutorials and traditional instruction. The study included state and privately run schools of different socioeconomic profiles, without formal laboratory space and equipment, in classes of mixed-gender and female-only students, taught by novice and experienced instructors. Results systematically show that student learning is significantly higher in the Tutorials classes compared with traditional teaching for all of the studied conditions. The results also show that long-term learning (one year after instruction) in the Tutorials classes is highly satisfactory, very similar to the performance of the samples of college students used to develop the test DIRECT. On the contrary, students following traditional instruction returned one year after instruction to the poor performance (< 20%) shown before instruction, a result compatible with the very low level of conceptual knowledge of basic physics recently determined by a systematic study of first-year students attending seven universities in Spain and four Latin-American countries. Some replication and adaptation problems and difficulties of this experience are noted, as well as recommendations for successful use of Tutorials in high schools of similar educational systems.


I. INTRODUCTION
Learning of science and physics, in particular, in high schools of Ibero-American countries is very poor and it is hampering socioeconomic development.This characteristic, which may be shared by education systems of other noncentral countries, has been repeatedly pointed out by the results of international surveys such as PISA [1], in which all participating Latin-American countries performed in science and mathematics in the lowest group.To reinforce and complement this picture, a recent study of first-year university students enrolled in science and engineering programs at seven universities in four Latin-American countries and Spain consistently showed that conceptual knowledge of physics is almost null [2,3].For instance, only about 7% of an overall sample of more than 3000 students beginning university science and engineering programs had a working conceptual knowledge of Coulomb's law, with similarly poor performances regarding the vertical motion of a tennis ball, Newton's laws, energy and force on an inclined plane, and simple resistive electric circuits.The situation was very similar, independent of country and type of university (private or state run).These results indicate that long-lasting conceptual knowledge is not the result of high school instruction and, furthermore, that precollege science education in this region is failing to meet basic goals, which result not only in unsatisfactory science and math preparation but, perhaps more important, in a lack of vocations for science and engineering and an ill-science-educated society [4].
While changing an education system is a complex and difficult endeavor that requires political decisions, recommending the most appropriate field-tested curricula and teacher-training strategies toward that objective is a central point that can and should be carried out by science educators.In this line of work, the main objective has been to determine if, under normal conditions of representative high schools of the Argentinean educational system, the use of physics education research (PER-) derived curriculum promotes higher levels of conceptual learning of basic physics.After considering several active learning curricular alternatives, it was decided to experiment with the applicability of Tutorials in Introductory Physics [5].We chose this curriculum because of its learning cycle, adaptability to different classroom settings and requirements, simplicity and economy of the teaching material, and for being available in Spanish [5].Throughout this paper, the word "Tutorials" refers to Tutorials in Introductory Physics and not to tutorials as this word is used more generally.
Tutorials in Introductory Physics is a product of PER [6], a field that has been very successful in continuously generating research-based, field-tested active learning curriculum that has been shown to be very efficient in fostering conceptual learning of basic physics [7].This pedagogical research and associated curriculum development have been mainly targeted at introductory physics courses for university students majoring in science, technology, engineering, and mathematics (STEM) largely in U.S. colleges and universities [8].Much less work has been devoted to pedagogical experiences with non-STEM populations, where motivation, interest, initial preparation, and other factors may influence student's learning outcome [9].For instance, Moore and Rubbo [10] reported recently on the application of interactive engagement teaching strategies in introductory physics and astronomy courses designed for nonscience majors.In particular, the physics course was based on Physics by Inquiry [11], a guided-inquiry approach developed for training future and in-service physics teachers.This and previous experiences [7] not only produced excellent learning gains, but also pointed out some differences between STEM and non-STEM students that seem to affect the learning gains [12][13][14][15][16].
Tutorials in Introductory Physics is one of the most replicated active learning teaching strategies, mostly in environments similar to the development site [6,8].These studies have shown the important increase in conceptual knowledge generated by the Tutorials, as well as their adaptability to different course environments, from supplementing rather traditional teaching to complementing other active learning strategies.Some studies also pointed out that replication always implies a certain degree of adaptation to the local conditions [17][18][19][20], and that successful implementations of Tutorials require respect for the learning model, an understanding that students must be intellectually active to build their own learning, and that the reasoning process is central to that learning.For these reasons, replicating research-based material such as Tutorials requires fidelity to the learning process and associated teaching material, which should not be modified unless careful research recommends modification for the local conditions.
Of the basic physics themes available in the Tutorials, we chose to focus on simple resistive electric circuits because it is this kind of conceptual and procedural knowledge with everyday applications that, in a modern society, every citizen should have.Furthermore, for the purpose of this longitudinal study, this subject is not followed or revisited in the science curriculum of local high schools.This characteristic should contribute to the independence of the present results and the instruction carried out between post-test I and post-test II.To allow for greater generalizability, private and state-run public schools with different resources, students' socioeconomic conditions, and gender were selected.We also invited two novice teachers to use the Tutorials in similar high school courses.In all cases, evaluation of conceptual knowledge of dc circuits was carried out by administering, before and after instruction, the multiple-choice, single-response test "Determining and Interpreting Resistive Electric Circuits Concepts Test" (DIRECT) [21].In the following sections the research questions and main aspects of the experiment are described, followed by the results, in which the methodology of analysis has been kept as simple as possible to facilitate the reproduction and/or comparison of the present experiment by interested teachers.

II. RESEARCH QUESTIONS
The aim of this study has been to support science education reform by providing objective information regarding the use of Tutorials in Introductory Physics, in representative high schools of the Argentinean educational system.A pre-and post-test comparison of intact classes was designed to compare the effectiveness of Tutorials instruction with that obtained by traditional instruction in local high schools.Conceptual knowledge of the subject matter, simple electric circuits, was evaluated with the multiple-choice, single-answer test DIRECT at different times in five selected classes in an effort to answer the following research questions: (1) How does conceptual knowledge of simple electric circuits change in a traditionally taught course versus a Tutorials-based course?(2) How do students retain physics understanding of simple electric circuits in both traditionally taught and Tutorials-based courses?(3) Are implementations of Tutorials effective for both experienced and novice teachers?

III. EXPERIMENT
Following the above research questions, the present experiment consisted of three complementary parts.First, we compared the outcome of traditional and Tutorials teaching in three classes taught by the same teacher, one control class (traditional teaching) and two experimental classes that used Tutorials.In the second part of the experiment, we looked for the long-term or residual knowledge.The aim was to find clues for the very poor conceptual knowledge of high school graduates detected by the Ibero-American survey cited above [2][3][4].This objective was accomplished by administering DIRECT, one year after instruction, to the same student samples of the first experiment.In the last part of experiment, we asked two novice teachers to reproduce, in the following school year, the experimental teaching in the same private and state-run schools.
Pre-and postinstruction testing of control and experimental groups allowed for comparison of five intact classes of the local educational system.In all cases, instruction ran for 3 1  2 weeks, with two class periods of 80 minutes per week.This total time included the time for the postinstruction diagnostic (post-test I).The subject matter included the concepts of resistance, current, and potential difference in series and parallel resistive electric circuits.

A. Student samples, teachers, and environment
Simple resistive electric circuits were taught to students attending the 11th year of instruction (16-17 years old) in high schools of San Luis, Argentina.One school is a staterun institution with a similar number of students of both sexes, coming from low-to middle-class families.There were two very similar physics classes in this school (called "divisions"): one was randomly selected as the control sample (hereafter Prof.A CRTL), while the other division (Prof.A-state) was considered the experimental group.Students were assigned to these two classes long before this experiment, following institutional rules.Except for the type of instruction, all educational conditions were equivalent (see Table 1SM of the Supplemental Material [22]): 30 students per class with no formal laboratory facilities or resources for laboratory material or equipment.The teaching materials (cables, batteries, and small light bulbs) were provided by some students and the teacher.This low-tech, low-resources environment is characteristic of the local school system.In the first part of this experiment, the experienced teacher (Prof.A) practiced both teaching approaches: traditional teaching in the control class and Tutorials teaching in the experimental division.In search of generalizability, we also implemented the Tutorials instruction in a private, female-only, school, run by a religious congregation (Prof.A-private).Students of this school, who mostly belong to middle-class families, have to pay a monthly school fee (instruction is free in the state-run school).To complement this picture, in the last part of the experiment, we asked two novice teachers (Prof.B and Prof. C) to conduct Tutorials teaching in similar courses of the same state and private schools, as shown in Table I.
Teacher A is an experienced teacher, with a long interest in improving his teaching practice.He followed a graduate program in physics teaching at the local national university and attended 18 professional development courses, two of them on Tutorials in Introductory Physics.Teacher B has just graduated as a high school physics teacher from an undergraduate program that includes three courses on different aspects of interactive engagement teaching, including the use of Tutorials.He also attended two short postgraduation workshops on active learning.Prof. C was a preservice teacher, who has taken the same three pregraduation courses on active learning as Prof. B, performing his professional teaching practice as part of this experiment.

B. Teaching approaches
Traditional teaching, based on lectures, which for this subject included simple experimental demonstrations.Lectures were complemented with problem-solving sessions and homework assignments.This practical work allowed students to apply concepts and the problemsolving approach exemplified by the teacher.Homework was evaluated and assigned credit.In-class students activities were essentially individual.
Experimental teaching, consisted of the application of two Tutorials: "Current and Resistance" and "Potential Difference," noted products of educational research [23,24].Selection of Tutorials as the experimental teaching approach was based on the learning cycle: elicit students' previous ideas, confront them with the outcome in the Tutorials, and resolve the differences.This cycle is implemented through three complementary activities, Tutorials pretest, Tutorials worksheet, and homework.These provide students with multiple challenges in different contexts as well as opportunities to apply, reflect, and generalize on the subject matter.
In the present application, extra efforts were made to replicate Tutorials teaching as closely as possible to the recommendations [25] of the developing site, overcoming important implementation problems.The most urgent problem concerned the total class time needed, since each Tutorials worksheet demanded about two class periods of small-group work.Students worked through the Tutorials worksheets in small collaborative groups of 3-4 members in the regular classroom.They had to move their individual desks so that they faced one another, building up in this way a small working table for circuit elements and paperwork.The homework assignments were evaluated by the teacher and also demonstrated by students, generating whole class discussions coordinated by the teacher.
Students were evaluated based on their homework and participation in the discussion sessions, but not for their answers to pre tests and Tutorials worksheets.Table 2SM of the Supplemental Material [22] describes main characteristics of the recommended [25] Tutorials teaching and of the present replication.

C. Measuring instrument
Conceptual learning was evaluated with the 29-item, multiple-choice, single-response test DIRECT.These items probe into 11 different particular learning objectives, which can be grouped into four general objectives: physical aspects of dc electric circuits, energy, current, and potential difference [21] (see also Table 3SM of the Supplemental Material [22]).The use of a research-based multiple-choice test allowed us to efficiently and objectively evaluate the performance of all participating courses, comparing them with previous and/or published results.DIRECT is part of a family of multiple-choice tests developed on the basis of results from research on learning difficulties and alternative conceptions, which are reflected in the distractors of the different items.Applied before instruction, this feature provides a "radiography" of students' learning difficulties, which can be used to program appropriate instruction.Application of DIRECT before and after instruction, besides providing the learning gain, can also be used to study the evolution of the students' difficulties and alternative models.Since not all subjects tested by DIRECT were covered by the instruction (traditional and experimental), we report, analyze, and comment only on matched-sample performances on the 19 items directly considered in the instruction.We excluded mainly those questions related to the energetic and microscopic aspects of dc circuits.Nevertheless, in all cases, the test was given in its complete form, and we used the ten items on subjects not covered by instruction to check for self-consistency.Finally, it is noted that we used a Spanish version of DIRECT 1.1 translated by one of us, which was validated by university professors teaching introductory and advanced electricity and magnetism.This version had already been used in experimental and regular teaching at local universities and high schools [19,26].Results are summarized in Table 3SM of the Supplemental Material [22], which follows the presentation by objectives of Table I of Ref. [21].
DIRECT was given at three different times: just before (pre-test) and after (post-test I, called post I hereafter) instruction, and one year after instruction (post-test-II, called post II hereafter).DIRECT items were never used in instruction in the five classes of this study, neither were they discussed in class by any of the teachers.It is also important to note that the subject matter (simple resistive electric circuits) was not revisited by instruction in the time between post-test I and post-test II.For this reason, we consider the results of the long-term evaluation (post-test II) as the enduring knowledge produced by the instruction evaluated in this experience.

A. Initial knowledge
Preinstruction (pre-test) knowledge of the subject matter was evaluated by administering DIRECT to all participating students just before instruction.
Table II shows that, in all cases, the average performance before instruction was slightly lower than the random response (20% for DIRECT).Since initial knowledge of the subject matter is also an important measure of the equivalency between courses, we decided to run a one-way analysis of variance (ANOVA) [27] test of all participating classes (including those of teachers B and C).Even though the test pointed out significant differences between courses (df ¼ 4, 144, F ¼ 5.483, p < 0.001), post hoc results revealed that only the sample Prof. A-state was statistically lower than the others.The other four samples were not statistically different among them.It is clear from the above results that the initial conceptual knowledge was essentially null in all tested classes, a characteristic of student groups without previous instruction.

B. Traditional versus Tutorials teaching
To answer the first research question, DIRECT was used to evaluate the conceptual knowledge immediately after instruction (post I) of the three courses taught by Prof. A. This allowed us to determine conceptual knowledge instruction gain G 1 ¼ post I − pre − test.
The average postinstruction performances of the three courses taught by Prof.A ranged from 39% for the control group to 63% and 68% for the two Tutorials courses.For comparison, the average performance of the compound student samples reported by Engelhardt and Beichner [21] is 49% for the same 19 items of the present experience.Table II also makes it evident that the large differences in post I-pre-test gain G 1 are correlated with the instruction mode, since both Tutorials courses more than double the gain of the control class.It should be added that there is a lack of post I-pre-test correlation for both experimental courses (r ¼ −0.08 and −0.11, respectively), while the control class shows a (modest) positive correlation (r ¼ 0.20).
Table II also shows the intrinsic or normalized gain of the averages obtained in these three courses.The intrinsic gain is defined as the fraction of the maximum possible gain obtained in a particular course, i.e., the ratio between the actual gain G 1 and the maximum possible gain (the difference between perfect postinstruction performance and the initial knowledge).
Therefore, g I ¼ ðpost I − pre-testÞ=ð100 − pre-testÞ.Tutorials courses show rather high normalized gains (g I ¼ 0.56-0.64),which more than double the modest gain obtained in the control course (0.24)

C. Long-term knowledge
To study the second research question, a longitudinal study was carried out to measure conceptual knowledge one year after instruction (post II).Long-term evaluation of student knowledge is difficult for various reasons, especially in college where student groups tend to disaggregate just after the course is finished.High school classes in our educational system, on the other hand, seem particularly suited for follow-up studies since the student groups remain almost intact.Furthermore, they all have to follow a common, fixed, and mandatory nationwide curriculum, with no further instruction regarding electric circuits.The post II results of the three classes taught by Prof.A are shown in Table II.The first striking result is that the mean performance of the control population returned, one year later, to the same precarious level of conceptual knowledge held before instruction, very close to the random answer.
The two experimental classes, Prof. A-state and Prof. A-private, clearly outperformed the control course, with overall performances of both experimental courses very close to the 49% performance of the better prepared population reported by Engelhardt and Beichner [21].
Even though the one-way ANOVA test pointed out statistically significant differences (df ¼ 2, 77, F ¼ 37.18, p < 0.001), post hoc results from this test showed that the mean post II performances of the two Tutorials classes were statistically different from the control sample but not significantly different between them.This finding is complemented by a statistical analysis that confirms a lack of correlation between the residual (post II) and initial (pre) knowledge for both experimental courses (r ¼ −0.20 and −0.16 for the Prof. A-private and state samples, respectively), while there is positive correlation for the control class (r ¼ 0.39).Tutorials instruction not only is very successful in improving overall, long-lasting conceptual learning, but this characteristic is independent of students' initial knowledge.On the contrary, traditional instruction seems to generate modest residual leaning only on those few students that showed some initial knowledge.This position is confirmed by linear regression analysis of posttest II as the dependent variable and pre-test and type of instruction (traditional or Tutorials) as independent variables, which show that post-test II results of students attending the Tutorials course were statistically different from those of the control course (traditional teaching).Moreover, the analysis shows that the type of instruction is the determining factor, while preinstruction knowledge is not a statistically significant predictor.
For instance, when comparing post-test II results of the control and private courses, the predictive power is quite good (adjusted R square ¼ 0.522), with a difference between groups of 26.698 (once the pre-test scores were controlled for) with an important size effect r ¼ 0.763 (see Table 4SM of the Supplemental Material [22] for test statistics).Similar results are obtained for the comparison between the control and state classes.
Table II also shows that the long-term intrinsic gain g II has dropped to 0.41 and 0.35 for the state and private experimental classes, respectively.This is a remarkable result, since, even with the natural drop of performance after a year of not being exposed to the subject, the longterm knowledge of students of the Tutorials courses still compares well with the results reported in the development of DIRECT [21].Table II also shows that one year after instruction the intrinsic gain drops by a very similar amount, Δg ∼ 0.20, independent of the type of instruction received by the students.The absolute performance, post-test II, drops by comparable amounts (∼20%) in all three cases.
In addition to the course average learning gains, a point of particular interest in our study concerns the effectiveness of instruction for any student of a given class.Figures SM1a and SM1b of the Supplemental Material [22] show the pre-test, post I, and post II performances of every student of the control and Prof. A-state classes, respectively.In addition to the large postinstruction mean performance differences between the control and experimental classes (the last three columns on the right), these figures visually point out that just a few students of the control class retain some conceptual knowledge one year after instruction, while most students of the experimental course still perform well.For instance, if we require a minimum of 40% of correct answers as an acceptable performance, one year after instruction, less than 20% of the control class met this requirement, compared with 73% of the state sample.Table III, which separates each sample by quartiles of the long-term gain G 2 , shows that just a very small fraction of students in the control class demonstrate a reasonable gain, while most students show very little (58%) or even negative gains (32%) (negative gains are interpreted here as another consequence of noncoherent answers by students holding rather diffuse alternative models).
The situation is exactly the opposite in the Tutorials classes, most of whose students are included in the quartile of maximum gain.

D. Effectiveness of experienced and novice teachers using Tutorials
Another point of relevance for science education reform concerns the ability of teachers of different experience in implementing active learning teaching strategies.To study this research question, in the following school year we asked two novice teachers (Prof.B and Prof. C) to repeat the Tutorials teaching in the same schools and courses of the original experience.Table II and the ANOVA test cited above showed that these two courses were, before instruction, equivalent to the courses taught by Prof. A. Table IV shows the students' performance of the five tested classes divided by quartiles of postinstruction gain G 1 .
The large differences between the control class and the four Tutorials classes, as well as the equivalent performance of the latter, are readily observed.Furthermore, the results show that the mean performance of the two courses taught by the less experienced teachers, B and C, were very close to those obtained in the courses taught by Prof. A.
Again, a one-way ANOVA test of the mean performances in post-test I found statistically significant differences (df ¼ 4, 142, F ¼ 12.698, p < 0.001), but the corresponding post hoc results revealed that all four experimental courses were statistically different from the control class but were not statistically different among them.

E. Gender performance
Learning of science by female students is less satisfactory in many education levels and systems, being, therefore, considered an important social and educational problem.Although this experiment was not designed to study the gender problem in depth, from the collected data we can get some insight into the problem by comparing the two female-only classes with the mixed-gender groups (which can be, furthermore, separated by gender).Table V shows the post-test II performance of the classes taught by Prof. A. It is readily seen that males and females of the control group are clearly outperformed by the males and females of the experimental samples, with similarity of results between the different samples that followed instruction with Tutorials.Although a slight difference can be detected between males and females of the same classes, post hoc test results from a one-way ANOVA analysis (df ¼ 4, 75, F ¼ 21.037, p < 0.001) of these five samples only detect statistically significant differences between groups of different teaching approaches, but do not detect statistically significant differences between male and female subgroups subject to the same teaching strategy (traditional or Tutorials).This seems to indicate that, under the conditions of the present experiment, satisfactory, gender-independent, long-term learning has been achieved by students in the two Tutorials classes.On the contrary, male and female students that followed traditional teaching essentially returned in post-test II to the very low knowledge level shown before instruction.

F. Some advantages and implementation problems using Tutorials
Shifting from traditional, teacher-centered instruction to the student-centered learning process advocated by Tutorials and other active learning methodologies implies a cultural change of the whole school community, not only students and teachers, but also school officials and parents.Previous replications of Tutorials [17][18][19][20] have noted that successful implementation of Tutorials requires adequate space, some additional resources, and, most importantly, additional and properly trained teaching staff.Teacher preparation for Tutorials instruction should be understood as a process in which the physics teachers should first work in small groups with other teachers on the big and small details of Tutorials worksheets.This is a mandatory first step for fully buying into the teaching approach.
In this implementation, the first problem was that each tutorial demanded much longer than the 50 minutes of class time reported at the development and replication sites [25].This obstacle also appeared in different implementations at the introductory level in local universities [4,19], where each Tutorials worksheet demanded between 2 and 2 1 2 hours to complete.In this experience, the accommodation achieved by replacing lectures with the Tutorials sessions and also limiting the time for exemplary problems solved by the teacher.We regard this extra difficulty as a manifestation of cultural problems derived from the poor communication and reasoning abilities of our students, already pointed out by international and local surveys [1,4].Although these characteristics certainly magnify the problem, it should be considered that active learning usually demands more time than traditional teaching and that timeconsuming activities, such as reasoning and small group discussions, are fundamental for the effectiveness of the teaching approach.For this reason, Tutorials teaching in our school system should be preceded by a rearrangement and selection of course content.Without this fundamental step, teachers will be forced to return to the "more effective" (less time-consuming) traditional teaching.
From the teachers' point of view, Tutorials propose ready-to-use activities, liberating them from preparing teaching activities, something that they usually are not prepared, nor have the time, to do efficiently.The present replication faced some additional problems: always being a one-teacher experience, the corresponding teachers could not benefit from the discussions and activities of the meeting of teaching assistants that is mandatory in the development and applications sites, prior to each Tutorials session [25].Another local problem is the high student-tofacilitator ratio, which is usually about 30∶1 against the recommended 12-15 students per teaching assistant.These local factors make it more evident that proper teacher preparation is absolutely mandatory for successful implementation of Tutorials.In the present case, adequate teacher preparation was achieved via two complementary actions: preservice instruction, through the three pedagogical content knowledge courses taken by the two novice teachers and through the teacher professional development short courses and workshops taken after graduation by Prof. A.
Working with Tutorials seemed to have also been satisfactory for students.They were always eager to carry out "hands-on" activities, participate in peers' discussions in the collaborative work proposed by Tutorials, as well as do the homework assignments.This is most important for our discipline, since physics is always regarded as a difficult subject, without connections with real life or practical matters.Students of both experimental classes of Prof.A manifested a high degree of satisfaction with the teaching strategy, but complained they have "to write too much."This discontent is probably a manifestation of the general resistance to educational changes, which is boosted in the present case by the very poor reading and writing abilities of local high school students [1].We are confident that the systematic use of active learning teaching strategies and Tutorials, in particular, can contribute to the development of reasoning and communication abilities, a problem that deserves deep and systematic studies in our school system.

V. CONCLUSIONS
The aim of this work has been to study the feasibility of using Tutorials under the particular conditions of the high school system of a noncentral country.In this framework, the two didactic units of Tutorials dealing with simple resistive electric circuits have been applied in five different courses, taught by three different teachers, in two high schools of different resources, socioeconomic profiles, and gender.Both schools have no formal laboratory facilities, and instruction was carried out in the normal classroom.In all cases, the conceptual knowledge before and after instruction was determined by application of the test DIRECT.
Before instruction, all five courses showed a very low mean conceptual knowledge, with predominance of alternative models, a characteristic of naive, uninstructed populations.
Post-test I results showed big learning differences between traditional and Tutorials teaching.The first result is that the two experimental classes taught by Prof.A obtained excellent normalized gains of the average, which more than doubled the gain obtained through traditional instruction.There were, furthermore, no statistical differences between the Tutorials courses; i.e., these results seem to be independent of type of school and gender.
Analysis of the long-term, residual conceptual knowledge (post-test II) showed that, one year after instruction, students of both Tutorials classes still performed at a satisfactory level, very similar to the mean performance of students of a more advanced and competitive educational system [21].Perhaps more important, this improvement is distributed among most of the students, independent of their initial knowledge.In this regard, linear regression analyses show that preinstruction knowledge is not a good predictor of students' learning, with the big differences between control and experimental courses determined only by the type of instruction.Analysis of the long-term learning gains shows that both experimental courses still show, one year after instruction, important normalized learning gains (g II ∼ 0.40), while the course that underwent traditional instruction returned, on average, to preinstruction knowledge, g II ∼ 0. While the latter result sheds some light on the very poor performance of first-year students attending seven Ibero-American universities [2,3], the measured drop in normalized gain, Δg ¼ g I-g II ∼ 0.20, similar for both teaching approaches, is a fact for which we do not have, at this time, a plausible explanation.It certainly deserves further study to determine if this "0.20 drop in g" one year after instruction is just idiosyncratic of our samples, subject matter, and measuring instrument, or a more general phenomenon.
The final part of this study strongly indicates that Tutorials also fosters excellent conceptual learning when novice teachers conduct instruction.It should be noted, however, that these novice teachers underwent appropriate professional preparation in active learning teaching and Tutorials, in particular.
In addition to the specific research questions, other points of general interest can be analyzed with the present data.The first is that long-term learning gains seem to be independent of gender, with courses of only females in a religious school performing similarly to the mixed-gender courses of the state-run school.Moreover, the analysis by gender of the latter course indicates similar accomplishments by males and females.
Another point is that postinstruction measurements on the control course illustrated some common features of traditional instruction.For instance, the whole course showed, immediately after instruction, some improvement in conceptual knowledge (post-test I), but one year later (post-test II) the mean class performance returned to the very low level measured before instruction.Furthermore, very few of these students achieved higher than 40% in post II, with most students performing very low, similar to, or even lower than before instruction.A widespread return to nonscientific models, with a multiplicity of prevalent alternative conceptions, was evident.A detailed analysis of students' answers showed, for instance, that most students in the control class were unable to individualize a short-circuited element in a realistic diagrammatic representation.
Poor results after traditional high school physics instruction do not seem to be idiosyncratic of the Argentinean school system.Similar results have been found in other countries by the Ibero-American study cited previously [2][3][4].For instance, they detected that more than half of the students were unable to identify short-circuited bulbs in realistic representations of simple resistive circuits.Although the low efficiency of traditional instruction in dealing with alternative conceptions has been already pointed out [8], the practically null effectiveness measured here could explain, at least in part, the surprising homogeneity of the very low conceptual knowledge of first-year university students detected by that study [2][3][4].
Although we have used DIRECT just as a measure of conceptual knowledge, the use of research-based, field-tested multiple-choice tests can provide important, in-depth information.For instance, analysis of the conceptual knowledge by objectives, as proposed by DIRECT [21], shows (see Table 3SM of the Supplemental Material [22]) that the objective "physical aspects of dc electric circuits" has been better achieved than the objective "potential difference."The former is associated with the Tutorials' "current and resistance," while the latter is the subject of the Tutorials' "potential difference."It seems that extending the basic model of current and resistance to the more complex one that includes the concept of potential difference is not a simple step for these students, a difficulty that could be related to the more abstract subject matter.Students seem more comfortable with concrete subjects than with more abstract ones, a reasonable feature shown also by non-STEM university students [10], but which could be even more relevant in the present experience, due to the very low level of scientific reasoning shown by local first-year university students [4].
Tutorials in Introductory Physics was developed for use in introductory university physics courses.Some of the Tutorials are also appropriate for high school students.Our experience shows that the Tutorials on simple electric circuits can be successfully used in high school instruction under conditions different from those of the development site.How general are these results?Our central concern has been to find ways to improve the conceptual learning of physics in high schools of the Argentine education system (and probably others of similar characteristics).As such, the chosen courses represent very closely the characteristics and conditions of local high schools.Although by the nature of our study we cannot claim a direct cause-effect relationship, the present results strongly indicate that Tutorials generate very important learning gains in students of different initial knowledge, gender, and socioeconomic conditions.They also indicate that these good results can be obtained in private and state-run schools, without special rooms or resources, with experienced and novice teachers, provided they are properly trained in active learning and Tutorials.This training should consider the importance of teachers buying into the teaching strategy [28], including emphasis on conceptual understanding and the importance of reinforcing peer discussion as a fundamental base for individual understanding.Teachers should understand that the (time-consuming) process of listening and constructing arguments to respond to others' ideas not only generates deeper learning but is central for developing reasoning and communication abilities.
Appropriate institutional support is central for educational change.The Tutorials replication reported here, which involved an important adaptation of local conditions, was fully supported by school authorities.A fundamental issue regarded the teaching time needed for Tutorials activities.This process implied adapting course contents, reducing or even eliminating some secondary subjects and applications, and emphasizing the more general concepts.The success of the present adaptation was certainly influenced by the experience with Tutorials of the authors [26].These facts highlight that teacher preparation is a central point that should be dealt with beforehand, when programming the curricular change.Another essential issue is that teaching reform should reach all related courses and teachers: we have already shown [29] that coherent instruction is fundamental for robust, conceptual learning.Students should not be subjected to teaching strategies that promote different values, as is the case of traditional and active learning instruction.
It is therefore clear that, also in our high school system, the use of Tutorials has great potential for supporting effective and substantial science educational reform in the area of physics, replacing traditional teaching by active learning methodologies.But, as a word of caution, we emphasize the central importance of institutional commitment, revision of course content, and proper teacher training as necessary and previous conditions for successful implementations of Tutorials in Introductory Physics.

TABLE I .
Characteristics of teachers and five student samples.# Courses refer to the number of professional developments courses and short workshops on active learning followed by each teacher.School Teacher Years of teaching No. of courses Teaching approach Student sample No. of students Student gender

TABLE II .
Mean (SD) performances (%) of the five classes of these experiments.Δpost ¼ post I-post II indicates the decrease in average class performance one year after instruction.Δg ¼ g I-g II is the corresponding decrease in normalized gain.

TABLE III .
Number of students by quartiles of the long-term gain G 2 (post-test II-pre-test) for the three courses taught by Prof. A.

TABLE IV .
Number of students by quartiles of the postinstruction gain G 1 (post-test I-pre-test) for the three courses taught by Prof.A and the courses taught by Prof. B and Prof. C.