Interactions between teaching assistants and students boost engagement in physics labs

Through in-class observations of teaching assistants (TAs) and students in the lab sections of a large introductory physics course, we study which TA behaviors can be used to predict student engagement and, in turn, how this engagement relates to learning. For the TAs, we record data to determine how they adhere to and deliver the lesson plan and how they interact with students during the lab. For the students, we use observations to record the level of student engagement and pre- and post-tests of lab skills to measure learning. We find that the frequency of TA-student interactions, especially those initiated by the TAs, is a positive and significant predictor of student engagement. Interestingly, the length of interactions is not significantly correlated with student engagement. In addition, we find that student engagement was a better predictor of post-test performance than pre-test scores. These results shed light on the manner in which students learn how to conduct inquiry and suggest that, by proactively engaging students, TAs may have a positive effect on student engagement, and therefore learning, in the lab.


I. INTRODUCTION
In science courses at the university level, teaching assistants (TAs) take on a variety of roles and are integral to the successful delivery of many courses. 1 These roles range from marking assignments and invigilating exams to facilitating laboratories and leading tutorials.
In large-scale courses, as are typical for first-year undergraduates at large universities, TAs are especially important for learning, 1 as in many cases they serve as the sole instructor in the smaller lab or tutorial sections. Despite the large amount of responsibility of the position, it is common that first-year graduate students, many without prior experience at teaching, are employed as TAs in these large courses.
In relation to student learning in these courses, the question becomes, what effect does the TA have on the students? Much work has been done on TA attitudes, with mixed results. For example, Goertzen and colleagues 2 studied how a TA's buy-in to reformed teaching methods affects classroom interactions in a reformed tutorial. Progress towards categorizing teaching beliefs appeared in work by Singer 3 and Trigwell. 4 More recently, studies have claimed a direct relationship between professed beliefs and teaching behaviors. 5,6 The connection between beliefs and behaviors is not so clear, however. As pointed out by Goertzen and colleagues, 7 different beliefs can lead to similar teaching behaviors. It has been advocated that a fine-grained analysis of a TA's behaviors and attitudes, using multiple sources of information is necessary to discern the true teaching beliefs of an instructor. 8,9 As evidenced here, there is some discussion as to the extent of the relationship between a TA's beliefs and their actions, due in part to the subjective nature of what defines a set of teaching beliefs. and student learning. From the perspective of the students, the beliefs and attitudes of the TA are not manifest, and instead are primarily communicated by the actions of the instructor. Then, studying the relation of a TA's behaviors to the students is a domain in which an effect would be expected. Specifically, we seek to evaluate the effect of the TA on students by studying which behaviors of TAs contribute to student motivation, engagement and learning.
A simplified model of the TA-student relationship that summarizes the above discussion is shown in Fig. 1. In this model, the TA's attitudes and beliefs influence their teaching style and behaviors in the classroom, which has an effect on student learning and attitudes.
As shown in the diagram, the dependence goes both ways; for example, the TA may update their teaching style based on the student response. As detailed above, much work has been done on TA attitudes and beliefs and the link between these and TA teaching style and behaviors. In this paper, we focus on evaluating the links that associate TA behaviours with student learning and attitudes. We note that there are many other factors, not included in this model, that influence both TA behavior and student learning and attitudes. For example, a TA may believe that a friendly and interactive style will benefit students, but their personality may cause them to behave differently in the classroom. As another example, it may be that a student is very motivated in a particular course due to a personal interest in the material and not because of any instructor's actions. Despite these limitations, this model conveys a simplified representation of the relationship between TAs and students.
We undertake this study in the lab sections of a large-scale first year physics course.
Within the inquiry-based lab, in which the TAs are the sole instructors, we look in particular at two aspects of the TA facilitation: how the TAs adhere to and deliver the lesson plan, and how the TAs interact with students during the work session. The study further evaluates two student factors: engagement and learning. Given that students learn in the lab by engaging in hands-on exploration, improving their engagement and reducing their off-task behaviors are key to improving learning. Measuring engagement has two additional benefits. First, these are direct, behavioral measures, that are not susceptible to students' interpretation and self-reports. Second, high engagement suggests high motivation, as students choose to engage with the tasks for longer periods. 14 Learning was measured by giving students an identical lab exam on the first and last weeks of the term.
The specific TA behaviors we observe (in italics), with our motivation and hypotheses for the relationship to engagement, are as follows: • How the TAs adhere to and deliver the lesson plan.
1. Number of announcements to the entire class. During periods in which the students are working on the lab material, the TA may make an announcement to the entire class, to clear up a common misunderstanding or to provide some instruction. One might expect that timely interjections from the TA might help students to have the tools they need to work through the lab, and therefore that this might have a positive correlation with engagement. We hypothesize that this type of interaction is not individualized enough to motivate the general student.
2. Behaviors outside of the standard TA script (excluding announcements to the class). In situations in which they are facilitating a class with a prescribed lab plan, TAs often add to the standard lab script. Examples of these extraneous behaviors include a dynamic classroom discussion involving both TAs, using the chalkboard during a discussion to illustrate a point, or soliciting student questions during a discussion. Since TAs add these events in an attempt to help the students, it might be expected that these extra behaviors are associated with student engagement.
• How the TAs interact with students during the work session.

Number of interactions.
Interactions between the TAs and students have a twofold effect. Primarily, they should motivate the students and help them to be interested in the material while giving them the tools they need to complete the lab work. Secondly, they should have a sort of 'policing' effect, in which a constant TA presence ensures that students are working on the lab. Therefore, we expect the number of interactions to correlate with engagement.

4.
Who initiates the interactions. The effect of the interaction may depend on who initiates it. 10 Interactions that are initiated by students may indicate that students are taking responsibility for their learning or they may reflect difficulties the students are having. These may be less dependant on TA facilitation style.
Therefore, to address our research questions, we are mainly curious about proactive interactions that were initiated by the TAs.

Length of interactions.
In general, the depth of the conversation between a TA and a student depends on the length of the interaction. During longer interactions, it might be expected that students' needs will more fully be met, as students get more attention and feel more invested in. Therefore, controlling for overall number of interactions, we expect that a tendency toward longer interactions will be correlated with student engagement.
In addition, given the nature of the inquiry-based lab, we hypothesize that engagement will be correlated with learning.

A. Design
We evaluate the effect of TA facilitation on student behavior by observing the lab. From an unobtrusive vantage point at the back of the lab room, observers were able to collect data on the TA behaviors listed in section I. By periodically circulating the room, observers were able to measure snapshots of student engagement in the lab. As discussed in more detail in section II C 1, strategies to minimize the disturbance of the observations on the classroom were utilized. The TA's behaviors in the lab are taken to be the independent variable and student engagement is the dependent variable.
The observations were completed during one lab session in a typical week in the middle of the term. Although observers measured behaviors and engagement in this particular lab, there is an assumption that the results can be extrapolated to the other lab sessions in the year. By the middle of the term, classroom norms, which include expectations for student engagement, have been firmly established by the TA's facilitation style. We expect the engagement levels we measure to be primarily a product of these classroom norms.
Since the lab we observed did not differ significantly in style from the previous labs, the TA behavioral data should be representative of the TA's behaviors in the lab throughout the term. Therefore, the results presented here are believed to be a reliable and generalizable subset of data that would have been obtained from observing the lab for the duration of the semester.

B. Participants and description of lab
The study took place in Physics 100, a first-year introductory physics course at the University of British Columbia (UBC), a large research-intensive university. 713 students were spread between 17 lab sections. The students in this course are primarily in the life sciences and, for most of them, this will be their only physics course. In addition, the majority of the students are first-year first-term university students.
The lab portion of Physics 100 has been extensively revamped in recent years to utilize an inquiry-based approach. The learning goals for the lab are aimed at the development of a general set of scientific and data analysis tools rather than being focused on content knowledge. A typical lab begins with an introduction and a set of clicker questions, followed by an extended period of students working in pairs, before ending with a summary discussion and another set of clicker questions. The average number of students in each lab was 39 while the largest and smallest sections had 46 and 25 students in them.
During the observation week, the main task of the lab was to extrapolate a data set in order to make a prediction for a future experiment. By measuring the time it took a coffee filter to fall to the ground from heights below 1 metre, students had to predict the time it would take for the coffee filter to fall from a height of 2 metres. To complete this inquirybased activity, students were required to design and carry out an experiment that would allow them to predict the falling time from the desired height, for example by performing a linear fit to data for various heights below 1 metre. Once each pair of students had made a prediction, one TA would perform (or get students to perform) the experiment at the height of 2 metres while the other would collect the predictions from each group. We define the 'student working period' in the lab as the time during which students are working to make their predictions, beginning after the introduction and ending once the TAs begin performing the experiment at a height of 2 metres or soliciting student predictions. The lab ended with a discussion comparing the predictions to the measured results.
In Physics 100, the TAs are the instructors in the lab, having full control of the section.
We exclude one of the lab sections from our analysis, because in the observation week there To measure student engagement, we developed the On/off task form (appendix C). This form consisted of a spatial map of the lab with empty squares to represent student positions at each lab bench. To fill out the form, the observer looked at the lab bench. If a student was on-task, as judged by the observer at a glance, a check-mark was placed in the corresponding square on the form while if a student was off-task, an 'x' was placed in the square. For example, a student that appeared to be writing on their lab worksheet was judged as ontask and a student that was looking at their cell phone or laptop was marked as off-task.
This form was completed at intervals of ten minutes during the lab, giving snapshots of the engagement level of the class. The fractional engagement for a snapshot is defined as the number of on-task students ( off-script TA behaviors (2.0 ± 0.8) were recorded and as such these categories did not show as good of an agreement. One snapshot of student engagement levels was taken, giving a fractional student engagement of 0.87 ± 0.06 between the observers, showing agreement of 93%. All disagreements between the observers were on two benches which appeared off-task by two of the observers but on-task by two other observers. Therefore, there is agreement within the observers on which benches were off-task and it appears that most of the variability in fractional engagement is due to the differences in the precise time at which the students were observed. We expect that such effects will be mitigated by recording multiple snapshots.
To record data on the TA observation form, the observers watched the TA from an unobtrusive vantage point at the back of the classroom. In order to not interfere with the lab, observers made a conscious effort to not talk or interact with the TAs or students in the lab. Further, the TAs were told only that the observers would be observing the classroom, and not recording specific data about the TA. We note also that these labs were often observed by visitors, and having additional people, who were unfamiliar to most TAs and students, was not perceived as any different from any other visitors. Then, the observer effect in the data should be minimized, and we assume that the TAs and students were able to act naturally. Due to the observation style, any data about the content and quality of TA-student interactions is outside the scope of this study. To complete the On/off task form required the observer to move from the back of the lab room, in order to properly observe the engagement of the students. Typically, the observer would take 1-2 minutes to walk around the classroom and fill out the form, again consciously avoiding interactions with students and TAs. To avoid the 'policing' effect, in which students in the lab might be motivated to appear to be working if they feel like they are being observed, the observers made an effort to observe the students discreetly. One strategy in this regard, previously used by Baker et al, 16 was to stand near one lab bench while observing a different bench.
To evaluate the relationship between engagement, interactions and learning, we evaluated student learning using pre-and post-lab exams. The lab exams were given during the first and last week of the lab. The exams were adapted from the Concise Data Processing Assessment 17 and Lawson's Classroom Test of Scientific Reasoning. 18 The tests were given without prior notice. All items were multiple choice, as shown in appendix D.

Analysis
The TA observation data was compiled to give data on a per section basis, as student engagement is expected to be a product of the behaviors of both TAs in a section.
The lab during the observation week may be broken down into four stages. Each lab began with a discussion of the previous week's homework (stage 1), followed by an introduction to the lab and the first set of clicker questions (stage 2). After the introduction, the 'student working period' portion of the lab began (stage 3). As defined above, this is the period in which students were working to predict the falling time of a coffee filter, when dropped from 2 metres. As students completed their predictions for the falling time, the lab moved into the final stage, in which TAs performed the 2 metre coffee filter experiment, discussed the results with the class and facilitated the concluding set of clicker questions. The length of time the lab spent in each stage is given in Table I. For the number of TA announcements, the number of TA-student interactions and the student engagement measurements, we use data from the 'student working period' only.
From the perspective of the TA, this part of the lab period is rather free-form. Within the main goal of supporting the students' activities, differences in TA facilitation style are expected to manifest themselves as different TA behaviors during this time. Further, it is this portion of the lab in which students are actively working, such that engagement is crucial for learning. Since the time each lab section spends in this 'student working period' varies, in order to compare across sections, the number of TA announcements and the number of TA-student interactions are normalized by the length of the section, giving announcement and interaction densities. The calculation of these densities takes into account moments in which observers were not observing due to a variety of reasons. (Usually, this was due to observers completing the On/off task form.) We restrict engagement results to this 'student working period' by taking only the first three student engagement snapshots recorded. These three snapshots span a time of 19.6 ± 3.3 minutes.
We recorded the number of behaviors outside of the standard TA script through the entire lab. Our hypothesis is that if such behaviors can be correlated with student engagement, it is the extra events that occur at all times in the lab, including during the introduction, conclusion and clicker questions, that contribute to student motivation. Since each lab section proceeds through the same agenda, so that each TA had the same opportunities to insert extra behaviors, normalizing by the length of the lab is not necessary here.
The length of interactions and the frequency of interactions are interdependent, as the longer the interactions are, the fewer interactions one has time to do. In order to disentangle the effect of the length of interactions from the number of interactions, we look at the ratio of short to long interactions, which gives a unit-less measure that does not depend on the frequency of interactions. Through the 16 sections, 581 TA-student interactions were recorded. Of these, 399 were less than a minute. Therefore, we have taken interactions lasting greater than or equal to a minute to be 'long', with interactions lasting less than a minute being 'short'. In this way, we can see if sections with predominantly long interactions are more engaged than classes with mostly short interactions.
As our unit of analysis is the section, we average test performance across all students in each section. Overall, students improved significantly from pre-to post-test. Pre-test: 66% ± 5%; post-test: 76% ± 4%. t(16) = 10.1, p < 0.0001.

A. Descriptives of TA teaching style and engagement results
We observed a high amount of variability between the 16 TA sections, as summarized in Table II. This wide variation in TA style is evident in the recorded number of behaviors outside the standard TA script (5.9 ± 2.6) and the frequency of classroom announcements (0.13 ± 0.14). The frequency of interactions across the sections was 1.31 ± 0.42. On average, 63% of the interactions were initiated by the TAs, 19 while the rest (except for a small number of interactions with no marked initiator) were initiated by students. Lastly, the ratio of short to long interactions was 2.8 ± 1.3; the majority of interactions were less than 1 minute long.
The student engagement results show a very high fractional engagement, at 0.88 ± 0.08, through the first three engagement snapshots.  We correlated our dependant variable (engagement) with each of the four independent variables, using an α level of 0.05. The number of interactions between TAs and students was a positive and significant predictor of student engagement: r(16) = 0.52, p = 0.03 (see Fig. 2). r 2 = 0.27, suggesting the 27% of the variability in student engagement per section is explained by the frequency of their interactions with TAs. As shown in Table III, the other three variables were not significantly correlated with student engagement.  Next, we studied how engagement depends on who initiated the interactions. The frequency of interactions initiated by TAs is significantly correlated with learning, r(16) = 0.49, p = 0.04. Interestingly, the frequency of interactions that were initiated by students is not correlated with engagement : r(16) = 0.052.

C. Correlation with learning
The frequency of overall interactions is correlated with learning, though insignificantly: r(16) = 0.24, p = 0.37. The measure of engagement is significantly correlated with learning: r(16) = 0.52, p = 0.03, r 2 = 0.27. Thus, 27% of students' performance in the lab can be explained by their engagement (as measured in a single session).

A. Discussion of results
Of the TA behaviors described in section I and monitored in our study, only the number of interactions per minute has a significant correlation with student engagement. Interestingly, the correlation between interactions and engagement is caused solely by TA-initiated interactions. A high rate of student-initiated interactions had no relationship with engagement, while a high rate of TA-initiated interactions is strongly associated with increased engagement.
While our observations are correlational and not causal, suggesting potential explanations for these correlations is of interest. Interactions may positively contribute to motivation and engagement by satisfying students' needs and enabling their success. Also, interactions may reduce disengagement due to a policing effect. The fact that interactions were related to engagement only when initiated by TAs suggests that the relationship is not due to student factors such as intrinsic motivations. Instead, it seems that TA behaviours are the ones that increase engagement, whether perceived as a carrot (due to the given assistance) or a stick (due to their policing effect).
Another possible explanation along these lines could be that the social aptitude of the TA is key for student motivation. A TA that is a social person by nature will certainly engage in more interactions with the students. In addition, these characteristics could allow the TA to be more aware of the students' needs and so provide better support to the students, as discussed for example by Scherr and Close. 20 In this scenario, the TA is the driver of student engagement and the determining factor is a combination of the number of TAstudent interactions and the quality of these interactions. Further study is necessary to determine if the quality of interactions is entangled with the number of interactions in this manner. One indication that this may not be the case is that the length of interactions was not correlated with engagement, even though we might expect that a more social TA would have longer interactions.
The dependency of the results presented above on the inquiry nature of the lab is of interest. We hypothesize that the correlation between interactions and engagement is not specific to an inquiry-based approach. Our two main inferences, that interactions allow students' needs to be satisfied and that the policing effect of interactions reduces disengagement, are expected to also be valid in a traditional style lab. Compared to an inquiry-based lab, the needs of the students would likely be different in a traditional lab. However, interactions would still allow the TA to support the students as they work through the lab.
Therefore, we expect the correlation between interactions and engagement to be generalizable across settings. In contrast, the correlation between engagement and learning may be specific to the inquiry-based lab. In such a lab, students are responsible for generating their own knowledge. If designed properly, there is no avenue for students to complete the lab without thinking about the process and reflecting on the results, as may be possible in a traditional lab. Thus, in a proper inquiry-based lab engagement should be closely correlated with learning while in a traditional lab the connection may not be clear.
It is interesting that the length of interactions does not correlate with the engagement of the students. Although we hypothesized that longer interactions are able to better satisfy the students' needs and help the students to feel invested in, according to the results here, a brief stop by the TA appears to be as effective as an in-depth interaction in keeping students engaged in the lab. A possible explanation is that by initiating a short interaction, the TAs open the door for deep questions from the students, in a sort of 'ventilation effect'. Thus, the length of the interaction is not an outcome of TA style, but rather, an adaptive behavior based on students' needs. In this case, the length should not affect engagement, as indeed observed in the data.
The number of class announcements does not correlate with student engagement in the lab. It may be that the positive effect of such announcements does not appear in student engagement or it may be that these announcements are not helpful to the students in this type of class. For the announcements to have a positive effect, the TA needs to have a good understanding of the needs of the class as a whole, and it is possible that this is difficult to determine in real time. We expect that a more focussed study is needed to determine if such announcements are helpful to students.
Despite the expectation that a TA that goes beyond the usual script may better inspire their students, we do not see a correlation with the number of these behaviors. It may be that our definition of this category was too broad and that only certain types of extraneous events have an effect on students. In that case, a more strictly defined 'extras' category, or set of categories, may be able to demonstrate the result.
In addition to the hypotheses tested above, our observations show interesting general results about how TAs spend their time in the class. We observed a large variability in the number of interactions, number of announcements to the class, and number of events added to the standard TA script. In addition to these, there was a substantial variation in the progression of each lab section. It is clear that there are meaningful differences in the facilitation style of different TAs and that these differences are an important factor in determining student engagement and learning. Further examination of the effects of TA's behavior on students is necessary in order to better understand which actions are beneficial to student learning.

B. Limitations and extensions
The main limitation of this study is our inability to access more fine-grained information about the interactions. It would be expected that the quality of interactions plays a large role in determining the student response to the TA. In addition, evaluating the content of the interactions would provide important data about which types of interactions are useful for student learning. To collect this data would either require the observers to be more intrusive, which would complicate the analysis, or necessitate the use of more advanced technology (i.e video and audio recording), both of which were outside the scope of our study.
The correlation of number of interactions with student engagement is an important first step in studying the effect of the TAs on the students in the lab. As an extension of this study, it would be interesting to correlate the TA's behaviors with student attitudes. This research will be technically difficult, as reliable methods for extracting the student response due to only one portion of a course have not been established.
An observer effect may be present in the student engagement data, as students may tend to make sure they appear to be working if they feel that someone is watching them. The average engagement level through 3 engagement snapshots is very high, at 88%, and the range is from 67% to 99%. Any observer effect would push engagement numbers higher and tend to minimize the differences in the data and wash out the effect. Thus, we expect that our method of collecting engagement data is sufficient and that the correlation between number of TA-student interactions and engagement is robust.

C. Implications for TA professional development
In addition to better understanding the effect of TA teaching style on student engagement, results from this study inform and support TA professional development efforts in order to encourage more productive tutoring styles.
The results presented here suggest that TAs should be active in the classroom and engage with as many groups as possible. Since the length of interactions was not found to affect the engagement of students in this course, it is possible that just saying 'Hello' may be as useful as a lengthy discussion in keeping students on task. However, a more likely explanation is that such short interactions are useful in keeping an eye on student needs and enabling a 'ventilation effect', giving students an opportunity to access the TA and opening the door for student questions that turn short interactions into long and meaningful interactions.

ACKNOWLEDGMENTS
We gratefully acknowledge Doug Bonn and Georg Rieger for useful discussions. This work was funded by the Carl Wieman Science Education Initiative.