Probing student understanding of scientific thinking in the context of introductory astrophysics

Common forms of testing of student understanding of science content can be misleading about their understanding of the nature of scientific thinking. Observational astronomy integrated with related ideas of force and motion is a rich context to explore the correlation between student content knowledge and student understanding of the scientific thinking about that content. In this paper, we describe this correlation in detail with a focus on a question about the relative motion of the Sun and the Earth. We find that high achieving high school students throughout New York City struggle with what constitutes scientific justification and thought processes, but can improve these skills tremendously in an inquiry-oriented summer astronomy-physics program.

Student understanding of both scientific content and scientific thinking is and has long been a priority in education.However, most of classroom instruction is typically devoted to content.With the emphasis on high stakes tests administered uniformly to large numbers of students at all levels, the focus on content has become more pronounced.Tests need to be administered broadly, reliably, and validly.As a result, questions are typically short answer, quantitative, vocabulary based, and cover a broader range of content than most students can meaningfully understand. 1 Little attention is given to "how do we know?" or "what is the explanation for?" or "does that make sense?"The results are that in many environments, students end up with a view of the nature of science and learning that is very different from what most teachers and experts want. 2 In this paper, we explore the nature of student scientific thinking with a cross section of students who have been through a variety of typical science education classroom experiences throughout New York City.We do not focus on their understanding of specific content, but rather on how they reason, how they try to make sense of scientific ideas, and how they explain and justify answers that they give.We feel that both instruction and investigating student ideas about science are best done within a specific context.For this study we have selected the context of observational astronomy and related ideas of force and motion.We believe the fundamental findings apply equally well to any other domain of science.
Student conceptual difficulties understanding topics in basic astronomy are well documented. 3,4For example, many believe that the phases of the moon are due to the shadow of the Earth or that the seasons are due to the varied distance between the Earth and the Sun.Understanding of these difficulties has led to the development of curriculum that helps students understand these astronomical ideas 5,6 which we have employed in this study.Also well documented are student conceptual difficulties with force and motion 7 including thinking motion implies force 8 and that forces of two interacting bodies are not equal. 9We have employed instructional strategies based on this research. 10yond conceptual understanding, introductory astrophysics is a rich context to explore thinking about science and the scientific process. 11,12How does one come to understand that the moon goes around the Earth or that the Earth goes around the Sun?How is it that some of those bright dots in the sky ͑that seemingly revolve around the Earth daily͒ are distant stars but others are planets going around the Sun? Models of the universe are not simply developed directly from observations.They need to be reasoned inferentially from extensive observations, integrating multiple domains of mathematics and physics.It is one thing to state the answers to these questions, but something very different to be able to understand the nature of science and scientific thinking that underlies how one comes to know the answers, which is the focus of this paper.
In the next section we describe the classroom context in which this study takes place and the backgrounds of the students enrolled.We then describe specifically how we investigated student thinking about science before giving a summary of our perspectives and implications.

II. CONTEXT
This research study takes place in the Summer Scholars Program at City College of New York, a summer enrichment program for students from New York City who just finished ninth through eleventh grades.The program is academically selective.Admissions requirements vary slightly from year to year but include a minimum of a B average and a letter of recommendation from the guidance office.Many of the students are from specialized high schools and are academically very successful.All have elected to spend their summers studying science.
The diversity of students in the program reflects the diversity of New York.Many are bilingual and come from families from, in order of frequency, Asia, Eastern Europe, and Latin America.Others ͑Asian, Black, and White͒ speak only English.The class is roughly half male and half female.
Curriculum and instruction are not predicated on covering a prescribed body of information or teaching toward a standardized test.The instructors have taken this as an opportunity to focus on the scientific process and depth of understanding of an accessible scope of content.This is in contrast to typical high school science instruction in New York City-and most other places.As described in the next section, the contrast results in a great culture shock for students who are normally accustomed to ͑and skilled at͒ succeeding on curriculum and assessment which focus on memorized facts, prescriptive problem solving, and multiple choice or short answer exams.
Each summer since 2000, 20-25 students have formed a science cohort.The program meets 4 days per week for 6 weeks starting in early July.A total of 4.5 h per day is formal class time divided equally between a chemistry class and an introductory astrophysics class, both delivered in the same inquiry-based spirit.This paper discusses only the astrophysics class.One of us ͑RNS͒ has been the lead instructor of the course since its inception.
Data presented are from the four summers of 2005-2008 and includes all existing data.Data from the earlier years are incomplete, but qualitatively there are no meaningful differences.

A. Qualitative observations
Due to the nature of the program and the dispositions of all instructors, classroom atmosphere is comfortable and open.Students are encouraged to question, challenge, discuss, and have fun.This atmosphere is conducive to getting to know the students well, both through direct discussions and through the inquiry-based classroom interactions.Every summer, both instructors and the teaching assistants gain insight into the evolution into student thinking, which is discussed explicitly.
The majority of students begin the summer clearly out of their comfort zone.They need to be given a great deal of prodding to provide reasoning, to negotiate what they are doing with each other, and to defend the arguments they make.They readily admit to seeing a science lesson as a body of information to be provided and written down and are surprised that the instructors prod them into figuring things out for themselves instead of just telling them answers.
As the summer progresses, most students grow increasingly comfortable with the nature of the class.Their skills with working with each other to answer their own questions increase.After repeated practice, they learn the difference between knowing something because they have been told it to be true and knowing something because they understand ͑and have often executed͒ the steps that underlie the idea.By the end of the summer such practices are largely selfdirected, typically with pride and self-awareness.

Pretest
Early in the first class session each summer, even before the course is defined, the pretest question shown in Fig. 1 is administered.All students complete the pretest independently and are given all the time they need.They are in an unfamiliar academic program with new classmates and all indications are that they try their best to answer the question.
As discussed below, this identical question is also asked at the end of the program.
Between 2005 and 2008, 93% of the students selected choice B on the pretest ͑N =83͒. 13This reflects agreement with the scientific community consensus.However, in this paper we do not focus on which of the multiple choice responses are selected.Instead we look in detail at what the students write in response to "provide a proper and complete scientific argument." Zhi 14 wrote as a scientific argument "The Earth goes around the Sun because of many reasons.One is the amount of time and days it takes for the Earth to go around the Sun.Another reason to account for this is the cycling of seasons we have in each year.This is why the Earth goes around the Sun."The student wrote clearly and in full sentences and made accurate references to relevant material.In a typical New York City high school class his response likely would result in a reasonable score and positive feedback.The problem is it does not address scientific justification for answering that the Earth goes around the Sun at all.Referring to the time it takes the Earth to go around the Sun as a reason that the Earth goes around the Sun is circular at best.The seasons can be accounted for in the geocentric and the heliocentric model equally well, so citing seasons as scientific justification for the Earth going around the Sun has no merit.
The scientific argument that Firoza provided was "According to Copernicus's geocentric theory the Earth goes around the Sun.Also the change of night and day shows that the Earth takes different positions and revolves around.Sun setting and Sun rising also changes our view of the Sun as we travel around it."As above, this is well written but without any substance.Like many students, Firoza referred to authority ͑Copernicus͒ and jargon ͑regardless of her choice of geocentric instead of heliocentric͒, neither of which constitute scientific justification.The rest of her answer demonstrates a lack of understanding of the relevant ideas as night and day have nothing to do with the conventional description of the revolution of the Earth around the Sun.
During class, after the pretests are collected, there is a whole class discussion.Student agreement that the Earth goes around the Sun ͑choice B͒ is the right answer is overwhelming, with many prefacing the choice with "of course" and "everyone knows."Explanations match those written on the pretest such as those described above, even as students are given opportunity and encouragement to elaborate.When instructor questions reveal holes in those explanations, students ask questions such as "Is it the seasons?" or "Does it have to do with the planets?" and then wait for an authoritative yes or no from the instructor, which does not come.This discussion is a prelude to the substance and philosophy of the course.While eager to have the answer to the seemingly simple question given to them, this is not done directly at any point during the program.
Zhi's and Firoza's answers are representative of many students.In order to quantify results, we developed a rubric specifically about the scientific arguments.Table I gives rubric scores, descriptions, and verbatim representative student responses.Each student response was scored by two reviewers.Out of the 166 student responses ͑83 pretest and 83 post-test͒, there were discrepancies in the scores of 28 of them and in each of these cases the discrepancy was only a single point.For these discrepancies, there was a discussion until there was a consensus on the final score.The average rubric score on the pretest is 1.37+ / −0.66.

Instruction
Both the instructional philosophy 15 and curriculum employed in the observational astronomy part of the course is largely from the Astronomy by Sight module of Physics by Inquiry 6 developed by the Physics Education Group at the University of Washington.Subject matter includes daily motion of the Sun, size, and shape of the Earth, phases of the moon, daily and annual motion of the stars, and motion of the planets.
Throughout, the emphasis is on the process of science rather than the presentation of facts.Students actively make observations and use these observations to construct and develop multiple scientific models of the universe.After making shadow plots of the Sun, students experiment with a flashlight and nail and reason that they can account for the daily relative motion of the Sun.After making and sharing observations of the moon, students observe the appearance of a ball near an illuminated bulb in an otherwise dark room.As before, students experiment and reason and come to recognize that they can account for all of their observations in multiple ways.Motion of the stars and planets are explored in a similar manner.Throughout, students are not given answers.Instead they are guided to develop ways in which they could account for their observations and are constantly asked to justify, explain, reason, and interpret.Given the unanswered question from the pretest, early in the program students repeatedly ask what the right answer is.As the summer goes though, there is a shift toward them wanting to answer the question for themselves.
In addition to the observational astronomy, students cover Newton's Laws of motion in a similar inquiry-based spirit.Included is a development of an understanding of gravity and the relationship between force and motion for circular motion.Near the end of the course, but prior to the post-test, the instructor covers important relevant historical observations such as retrograde motion, phases of the planets, and the moons of Jupiter.Implications of these observations and Newton's Laws are alluded to, but still not stated outright.Students are guided to making relevant connections on their own.

Post-test
At the end of the summer, the identical question shown in Fig. 1 is included on the final exam.Student responses are evaluated with the same rubric in Table I.The average rubric score on the post-test went up to 3.90+ / −1.05 from the pretest score of 1.37+ / −0.66.Table II shows several matched student responses along with the rubric scores.
Like many students, Venkat starts the summer with a combination of authoritarian statements and circular reasoning to support his choice of B. ͑See Table II.͒By the end of the summer, he still answers that the Earth goes around the Sun, but he now recognizes that the basic observations can be accounted for in either the heliocentric or geocentric model.To support choice B that the Earth goes around the Sun, he integrates Newton's laws and appropriate, more subtle observations of Mars.
Hariti also selects choice B at the beginning of the summer.͑See Table II.͒Her support for this choice is difficult to follow and is based on irrelevant and incorrect observations and reasoning.Her response reflects many students' written and later spoken justifications which are convoluted and stated confidently.By the end of the summer, Hariti changes her answer to choice D. In contrast to her pretest, her justification is clear and relevant.She properly justifies her choice by noting how it is possible to account for her observations in either model of the Earth and the Sun.
Curiously, student postcourse responses such as those given in Tables I and II reflect statements that were either changed somewhat from what was presented in class ͑Mars being close in a telescope͒ or refer to topics that were never covered ͑relative size of Jupiter and its moons͒.We are not sure whether this should be interpreted as enthusiastic students pursuing information outside of class or as evidence of the lack of effectiveness of didactic presentations.
Table II describes scientific justification.We also have data on the percentage of students who selected each of the choices A through D in Fig. 1.Of the 83 students, 93% selected choice B prior to instruction.On the post-test, 52% selected choice B and 43% selected choice A. One can interpret this as instruction having the opposite effect of improving success since many students switched their answer from the scientific community consensus answer to "I do not know."Instead, this result highlights the limitation of multiple choice questions which emphasize recall of what was told to students."Correct" responses on the pretest mask student lack of understanding of underlying scientific thinking.Explained "I do not know" responses on the post-test reveal student insight into well developed scientific reasoning.
The split responses between A and D on the post-test are consistent with the class approach of having students figure out scientific ideas for themselves.Students are given opportunities to make observations and build scientific models.While support and guidance are provided, answers are not stated authoritatively at any point.In the end, roughly half of Pre, B "The Sun has a larger mass than the Earth.With all masses, they exert an equal and opposite force on each other, this mutual gravitational force is exerted on both the Sun and the Earth.However, the acceleration of an object is inversely proportional with its mass.Because of the Sun's larger mass, the Earth accelerates around the Sun." Post, A "We can perfectly account for all our observations w/this model: seasons, phases of the moon, planets going around and moving backwards and forwards etc. but there is one characteristic that shows that this is the best choice: WE DON'T MOVE!Although proponents of the alternative view say that this model is so complicated and will get less migraines, they still not been able to explain why we don't feel ourselves moving.Also, just because it's complicated doesn't mean it's wrong, we must accept what is there is not a fantasy that is easier.The Sun goes around the Earth, no doubt about that." the students are convinced that there is sufficient evidence to support that the Earth goes around the Sun and justify their answers accordingly.Most of the rest of the students remain unconvinced and appropriately explain why.Either way, student responses reflect their own thinking and are connected to legitimate scientific thought in contrast to parroted, not understood answers.

Post-test 2
In addition to being asked to provide a scientific argument about the relative motion of the Earth and Sun at the end of the semester, students are asked the question about black holes shown in Fig. 2. The Earth-Sun question addresses material explicitly covered in the class, even though the question is never answered directly.In contrast, the black hole question addresses material not at all covered in the class.
Most of the students ͑95%͒ selected choice D, "I do not know."To gauge student scientific arguments, we developed and administered a similar rubric to that shown in Table I.The average score using this rubric is 3.85+ / −0.98, similar to the average score on the Earth-Sun post-test.
Many of the quotes on the black hole post-test reflect the epistemology emphasized in the class.Maria wrote "I have absolutely no idea whether black holes exist or not.I have heard of them mentioned, but I never really learned about them, so I really don't know…" Patricia wrote "… Since it isn't something I learned based on observations or experiments, I can only say that 'I do not know' unless I wish to spit out information I don't fully understand."Many were more explicit about referring to what they had learned in class such as Swatti who said "If I hadn't taken this class I would have said A, but now I know that I don't have anything to justify that other than 'I read it in a book.'" The scientific arguments of Maria, Patricia, and Swatti reflect a strong perspective of the origin of scientific knowledge.However, since they all selected choice D it also represents a potential limitation of their learning science content after having participated in this program.We certainly want our students to have an understanding and perspective of science which extends beyond that which is constructed from first principles and personal observations.To reach higher levels of understanding of science, students should be able to recognize how scientific ideas are constructed, when they are able to construct those ideas for themselves, and when they should accept the findings of others having gone through similar steps.

C. Epistemological survey
The pretest, post-tests, and qualitative observations described above give an indirect perspective of student beliefs about the nature of knowing and learning science.Another approach to probing these epistemologies is through a more direct survey, such as CLASS, 16 EBAPS, 17 VASS, 18 or VNOS. 19We wanted to determine how students in this study describe themselves in such a survey and how their responses correlate with our other epistemological measures.Post, B "Based on our observations, we can account for the daily motion of the Sun, stars, and planets using both models ͑geocentric and heliocentric͒.If we strictly rely on this as far as we can tell we do not know.Using our knowledge of physics, we may find evidence that gives the heliocentric model preference over the geocentric model.We know that the size of the Sun is much greater than the size of the Earth.͑The distance between the moon and the Earth is much greater than the size of the Earth and in pt 1 we showed that the distance between the Sun and the Earth is much greater than that, but yet the Sun appears to be the same size as the moon in the sky therefore the size of the Sun is much greater than that of the moon and of the Earth͒ By Newton's third law the force that the Sun exerts on the Earth is the same exact force, opposite, with the same magnitude that the Earth applies on the Sun.In order to maintain circular motion, at as constant speed, k, the direction of the force must be towards the center of the circle ͑Earth/Sun͒.If the geocentric model was true, the acceleration that the Earth has must be much less than that of the Sun.This implies that the Earth's mass is much greater than the Sun so the Sun can orbit around Earth.However we know that the m_EarthϽ M _ Sun.B "Before Newton's Laws were introduced, both A and B would have been possible.If A were the case, then the Sun would move clockwise around the Earth, and if B were the case then the Earth would move counterclockwise around the Sun.Both would account for the same conditions.However Newton explained that the more mass an object has the more gravitational pull.From our observations the Earth has less mass than the Sun, so the gravitational pull is greater.Also we discussed Mars coming closer and the only way that could be accounted for is if the Earth and Mars orbits around the Sun, and as they orbit the distance between them changes.The laws of motion and force support that the Earth goes around the Sun." 4 Hariti B "The Earth goes around the Sun because the different hemispheres of the Earth receive the Sun at different angles at different times of the day.The Sun, however is always in the same position when it is visible.Therefore, the Sun does not change position, rather, the Earth does." 1 D "The relative motion of the Earth and the Sun can be accounted for in both ways.Through ray observation.I saw both ways to be accurate.I saw that the Earth can go around the Sun counter clockwise and account for the relative motion of the Sun and the Earth.I also saw that the Sun can go around the Earth clockwise and it still would account for the relative motion of the Sun and Earth.There is no reason to choose another model using only the Sun and Earth's relative motion.Therefore, I do not know which model is better because with this information, the results are reproduced with the same amount of accuracy." 5 Xin B "This can be explained scientifically.The sun is a much larger planetary body therefore exerting a stronger gravitational pull on earth than the earth does on the sun therefore it's impossible for the sun to rotate around the earth." 2 D "From our ͑human͒ point of view, the path the Sun takes across the sky can be explained by either A or B. I know this because I observed with my own eyes both a helio and geo centric model.Repeated experiments show the same path of the Sun for either model.Not only that, but the path of the moon and phases also be accounted for by either model.In the helio model, the moon may be either revolving around the Earth clockwise or the Earth can be revolving around the moon counterclockwise to account for the apparent path/phases of the moon.In the geocentric model, the moon must be revolving around the Earth to account for the apparent path/phases of the moon.Not only can either model explain the daily motions of the Sun and moon, but both can also explain the annual motions of the Sun and stars.GEO:-Sun revolves around the Earth in and up-and-down orbit, or with the Earth on a tilt ͑in respect to the Sun's orbit͒-stars revolve around the Earth on a "sphere" far from Earth.HELIO:-Earth revolves around the Sun in an up and down orbit, or on a tilt ͑in respect to it's orbit-stars are fixed very far from the Earth, explaining the annual observations we see of the stars͒ Since both models account for all our observations well, we cannot prove or disprove either model to be correct or incorrect."

5
Students are asked to complete the survey shown in Fig. 3 at the beginning and end of the program.The items are based on MPEX ͑Ref.20͒ and probe the epistemological dimensions described by Hammer. 21These dimensions are related to student beliefs about their approaches to science.Do they think of science as a body of information to be provided by authority or as something that they can learn for themselves?Do they think that science has a conceptual foundation or do they rely on formulas without regard to whether they understand them or utilize them correctly?Do they think that science is connected to the real world or specific to the science classroom?We interpret survey responses as an indication as to whether students think they approach understanding science in a way consistent with what is expected by experts.
For data analysis, all student survey items are converted to a 1-5 scale with 1 being unfavorable and 5 being favorable.For survey items where expert responses are to strongly agree ͑items 5 and 16͒, no change was made in the student selection.For survey items where expert responses are to strongly disagree ͑the rest of the survey items͒, student selections are inverted, that is a survey response of 1 ͑the favorable strongly disagree͒ is converted to a 5 and 5 ͑the unfavorable strongly agree͒ is converted to a 1.In this way all rubric and survey data presented correspond to 5 being favorable.
Data suggest that the majority of the students begin class believing they have a favorable approach to understanding science ͑average responses of all 17 rescaled survey items 3.77+ / −0.39͒ and that it goes up slightly after the program ͑3.94+ / −0.42͒.This is in contrast with what we infer based on the way they perform in class, particularly on the pretest.This is discussed further below.

D. Instructor evaluations
As described above, instructors have great opportunity to have insight into the ways students approach doing science through the summer.We get to see how they justify their answers, how they reason through unfamiliar problems, how they respond when they do not know the answer, and how they interact with their peers in genuine problem solving opportunities.Each summer, before looking at the data described above, the instructors attempt to quantify student epistemology on a one to five scale, with five representing the most favorable epistemological dispositions described in the previous section and one representing the least.These were qualitative estimations based on the entire summer and were averaged among all instructors rating the students.
The average score for all students is 3.66+ / −0.80 out of 5. Despite involvement in the summer program, 21% of students were rated 3.0 or lower.These students averaged a score of 3.59 on the Earth-Sun post-test as opposed to a score of 3.97 for those who were rated higher than 3.0.

E. Correlations
In the sections above, we describe three different methods for investigating student thinking about the nature of science: student reasoning in explaining the answer to a scientific question, student self-assessment of their own epistemologies, and inferences drawn from qualitative observations by the instructors in an inquiry-oriented learning environment.Figure 4 shows the overall averages.
We calculated correlation coefficients for the three techniques for the full set of all matched data ͑N =80͒.The correlation table is shown in Table III.The variables correlated are Earth-Sun pretest rubric score, Earth-Sun post-test rubric score, precourse epistemological survey score, postcourse epistemological survey score, and instructor evaluations.The pre-and post-Earth-Sun rubric scores refer to the question in Fig. 1.The epistemological survey score and the instructor evaluations score are as described above.
While each research method probes how students approach science, they give unique perspectives.Since student thinking about science content is fragmented 22 and their epistemologies context dependent, 23 it should not be surprising that the results for each student are not identical for each of the three measures and that the correlations are low.
Although the sample size is small, Table III shows a low correlation between the epistemological survey and all other measures and a relatively high correlation between instructor evaluations and the Earth-Sun post-test score.We believe that many students are answering the survey differently from the way they behave and that instructors are valuing in-class approaches that lead to strong explanations on the post-test.

#
I t e m S c a l e Often, a scientific principle or theory just doesn't make sense.In those cases you have to accept it and move one because not everything in science is supposed to make sense.For each item, students were asked to circle a number between 1 ͑strongly disagree͒ and 5 ͑strongly agree͒.

IV. IMPLICATIONS AND CONCLUSIONS
The results presented in this paper are from a dedicated and academically successful population and suggest that after traditional instruction that even they struggle with important and basic understandings about the nature of scientific thinking.In the last ten years, we have administered similar instruction and research questions with hundreds of other middle school and high school science students and teachers as well as college science students.The results are fundamentally the same as what is reported in this paper.Future teachers are taught science in high school and college in a way that does not promote the understanding of the nature of science and then they teach science the same way.
If the question in Fig. 1 were a standardized test question, the 93% "correct" response rate would be interpreted as high success.Students and instructors would be lauded.There would be a march to more advanced material where it would be inevitable that students would have an even harder time understanding the nature of science and the underlying con-cepts and would therefore revert further into a strategy of memorize and repeat.Only after some probing does it become clear that student understanding is inadequate.Even if this probing is more challenging than a typical standardized test can accomplish, it is important if we are to emphasize a meaningful understanding of science.

FIG. 4 .
FIG. 4. Average scores for all research methods used in this study.

TABLE I .
Rubric used to evaluate student responses to scientific argument question shown in Fig.1.The sample answers are verbatim and representative."Pre"and "Post" refer to whether the response was given prior to or after instruction.A, B, C, and D refer to the multiple choice response given to the question.The heliocentric theory holds.The works of Tycho Brahe and Johanes Kepler and Galileo prove that the Earth does revolve around the Sun.Books: Kepler on the revolution of mass, Galileo Dialogues of the two chief systems.The Earth revolves around the Sun and not the other way around.It better explained 'epicycles' that Aristotle reasoned to account for the irregular orbits of other celestial objects observed from Earth.The Earth is proven to revolve around the Sun through extensive mathematical calculations performed by scientists in the past centuries" Pre, B "The Sun is the center of our solar system.All nine planets revolve around this star.We know that the Earth revolves around the Sun because we have night on one side of the Earth, and day on the other.The changing of the seasons is also a result of the Earth revolution."Pre, B "The Earth turns on its own axis while following an elliptical orbit around the Sun in which at some points it is either closer or further from the Sun.This full path around the Sun is the duration of one year.The Sun also spinning."I don't know, in some classes you are informed the Earth goes around the Sun, proof by scientific observation ͑through the change of seasons, shadows of an grounded object etc͒, however, like all scientific theories, you never know if it is true all the time ͑someone can always find another plot to say it is wrong͒.A few hundred years ago, people use to think the Sun goes around the Earth, so I really don't know what's the true answer of this question."Post, B "The Earth goes around the Sun because from the observations I made about the other planets going around the Sun.The planets are much smaller than the Sun so they revolve around the Sun.According to Newton's law F = ma and smaller things revolve around bigger things.Since the force between the Sun and the Earth is the same and the mass are very different.It means that the Earth will have a lot more acceleration than the Sun since Earth is little compared to the Sun" Pre, B "I believe the Sun has greater gravitational force than the Earth does.So the Sun pulls in the Earth and the Earth has no chance to move around the Sun" Pre, B "The Earth goes around the Sun because the Sun has a greater gravitational pull than the Earth does.Therefore, rather than the Sun being pulled into the Earth's orbit, the Earth and the rest of the nine planets get pulled into the Sun's orbit" Pre, B "The Sun's gravitational field is stronger than the Earth, thus the sun can keep the Earth in orbit due to enough centripetal force.While the sun does not revolve around Earth because Earth's gravitational pull on the massive sun does not produce

TABLE I .
͑Continued.͒ So the Earth revolves around the Sun" Post, D "The relative motion of the Earth and the Sun can be accounted for in both ways.Through ray observation.I saw both ways to be accurate.I saw that the Earth can go around the Sun counterclockwise and account for the relative motion of the Sun and the Earth.I also saw that the Sun can go around the Earth clockwise and it still would account for the relative motion of the Sun and Earth.There is no reason to choose another model using only the Sun and Earth's relative motion.Therefore, I do not know which model is better because with this information, the results are reproduced with the same amount of accuracy."

TABLE II .
Sample verbatim matched pre-post-responses to scientific argument question shown in Fig.1.Pretest and Post-test refer to prior to after instruction.A, B, C, and D refer to the multiple choice response given to the question.

TABLE II .
͑Continued.͒ feel that the Earth goes around the sun because of the Sun's gravitational pull that it exhibits on the Earth and the other planets.Most things can be explained in both the heliocentric model and the geocentric model.However some things does not go around the Earth like the planets.Sometimes Mars can be relatively close under a telescope and other times it is extremely far.The planets also does not follow the same pattern as the stars as seen from the Earth.If you put these observations on the heliocentric model, it works.We also discovered that the moons of Jupiter revolved around Jupiter in a circular motion.We observed that the small things tends to revolve around bigger things because Jupiter looked 200 times bigger than the moons that was around it.From that observation and others, like our moon revolving around the Earth, I conclude that the Earth goes around the Sun." D. I do not know.As best as you can, provide a proper and complete scientific argument for your answer.FIG. 2. Post-test question given to all students at the end of the summer program.Black holes were not covered.

1
When real life experiences differ from what is learned in a science text book the real life experience should be ignored in order to learn the science well.

TABLE III .
Correlation table among Earth-Sun pretest and post-test rubric score ͑see Table I͒, epistemological pre-and postscores ͑See Fig. 3͒, and instructor evaluations.