Instructor perspectives on the emergency transition to remote instruction of physics labs

Laboratory courses are an important part of the undergraduate physics curriculum. During physics labs, students can engage in authentic, hands-on experimental practices, which can prepare them for graduate school, research laboratories, and jobs in industry. Because of the COVID-19 pandemic in Spring 2020, colleges and universities across the world rapidly transitioned to teaching labs remotely. In this work, we report results from a survey of physics lab instructors on how they adapted their courses in the transition to emergency remote teaching. We identified three common themes in the instructors ’ responses: (i) using a variety of simulation tools, (ii) changing learning goals of the courses to be more concept focused, and (iii) reducing group work due to equity and technological concerns. We discuss the common challenges and successes reported by instructors, which leads to themes and lessons that can impact future remote and in-person instruction.


I. INTRODUCTION
As the COVID-19 pandemic began in the spring of 2020, instructors at colleges and universities worked quickly to move classes and activities online. Many students were suddenly forced to leave their campus homes-facing loss of employment, health concerns for themselves and their family, as well as navigating a new modality of learning. Likewise, instructors had limited time to determine new activities, struggled with ethical considerations of emergency remote instruction, and had to learn how to use new technologies, all while handling the impact of COVID-19 on their own personal lives [1,2].
To understand what happened during this switch to emergency remote teaching in physics lab classes, we developed an online survey to gather information from instructors after the Spring 2020 semester. This "instructor survey" was completed by over 100 physics laboratory instructors, mostly in the United States. The survey contained both closed-and open-response questions, which asked instructors about their experience transitioning to remote lab instruction.
Although the exact nature of the rapid transition was unprecedented and unlikely to occur regularly, many schools are now exploring remote alternatives when there are disruptions to teaching due to natural disasters such as fires, hurricanes, and snow-all of which would result in the need to rapidly adapt labs online for short periods of time. Likewise, as demands for online education have grown, so too are our needs to study these environments and determine effective practices in remote instruction. Although many of the choices made by lab instructors during Spring 2020 derived from necessity and overwhelming constraints, we were surprised by the number of survey respondents who discussed the successes and things they hoped to continue practicing beyond the pandemic circumstances. In addition, many instructors reported missing staples of physics labs, such as group work and hands-on experiments, which may spur a renewed emphasis on these aspects during in-person labs.
In this work, we expand on the initial quantitative results presented in a report posted to the arXiv preprint server in July 2020 in order to disseminate the relevant information as quickly as possible to the community (see Ref. [1]). In the initial report, we found that the transition presented particular challenges for laboratory course instructors, whom often rely on hands-on activities in a complex, collaborative environment involving various technical equipment to help their students learn experimental physics.
In contrast, this work uses a mixed-methods approachcombining statistical analysis on closed-response data with qualitative analysis on rich, open-response data-to identify common themes, challenges, and successes that instructors experienced. We answer the following research questions: RQ1. What were common themes surrounding the transition to remote instruction of physics labs expressed by the instructors? RQ2. In what ways, if any, can the transition to remote instruction inform lab course design for both in-person and remote labs in the future? The results from this survey can be used to help motivate education researchers to further study the opportunities and limitations of remote labs. Here, when we use the term remote labs we include all continued instruction of a course that was considered a lab course prior to the rapid transition to remote work, and in which the instructor and all students were no longer present at the same location [3]. Presenting different approaches to remote labs will increase instructors' knowledge of creative practices that could be used for lab courses both during an emergency and outside of such an event to increase opportunities for students generally, as well as those with limited access to in-person instruction.
We begin this work by presenting relevant background on research studying virtual and remote labs prior to the COVID-19 pandemic, in addition to contemporaneous studies on the impact COVID-19 has had on physics and lab education (Sec. II). In Sec. III, we provide the methodology for our analysis including our survey administration and design, analysis methods, and qualifications of our results. We structure the results and discussion section of this work, in Sec. IV, around three themes: (i) using a variety of simulation tools, (ii) changing learning goals of the courses to be more concept focused, and (iii) reducing group work due to equity and technological concerns. Additionally, in Sec. IV D, we reflect on the impact the transition to remote instruction may have on future education practices and draw conclusions from this work in Sec. V.

II. BACKGROUND
Previous research on the effectiveness of remote lab work has resulted in inconclusive findings, with strong advocates for both traditional hands-on labs and nontraditional approaches [4][5][6][7]. The differences in opinion on remote labs are often attributed to differences in learning goals and objectives between instructors and assessment tools. For example, proponents of hands-on labs often value design skills and social interaction [4], while proponents of remote labs often value learning content and theory [7]. Other possible benefits of remote lab experiences include providing more flexibility [8] and increasing accessibility for students who are part-time, have disabilities, or have caring responsibilities [9].
Unlike these past studies that considered intentionally designed remote labs taught by instructors with prior experience navigating an online teaching environment, our work focuses on the unforeseen, urgent, and stressful transition to remote learning due to the pandemic. One of the first decisions instructors had to make was whether to teach synchronously or asynchronously. Synchronous online classes would allow courses to more closely resemble the in-person experience, but could create inequitable classroom experiences for students struggling with technological limitations, new personal responsibilities, or other issues during the pandemic, such as being in a different time zone. Since the start of the pandemic, there have been several studies [10][11][12][13][14][15][16][17][18][19][20] that looked at the impact of these types of decisions on physics and science, technology, engineering, and mathematics classes, and even among these studies there are contradictory findings that speak to the complexity, and the highly contextspecific nature, of these decisions-a common theme we saw throughout our work.
For example, in the case of synchronous versus asynchronous instruction, a study by Wilcox and Vignal suggests that there was no difference in student perceived effectiveness for synchronous versus asynchronous lecture formats in their survey population [10]. However, in a study by Guo [11], which looked at a single physics SCALE-UP [21] style class, they found students who attended the synchronous sessions had an average test grade drop from prepandemic of 3.5 percentage points, while students who did not attend had a drop of 14.5 percentage points. In addition, the survey showed that students who did not attend the synchronous sessions found the course more difficult and felt they spent more time on the class than those who attended [11]. Likewise, a prepandemic study by Faulconer et al. [22] found that the withdrawal rate was higher from online introductory physics courses than their in-person equivalent, but those who persist in the online version were more likely to achieve a grade of "A" than in other modes. Additionally, they found that students enrolled in an environment where there is a visible peer support were less likely to withdraw from the course.
In a comparative study of the impact of remote physics lab instruction on student views about experimental physics including over 3200 students, Fox et al. found that there was no difference in student overall scores on the Colorado Learning Attitudes about Science Survey for Experimental Physics (E-CLASS) [23] when comparing courses from both Spring and Fall 2020 with the same courses in Spring and Fall 2019 [12]. Likewise, a study by Rosen and Kelly prior to the pandemic, found that there were no differences in students' epistemological beliefs about experimental physics between the in-person and the online lab [14]. However, there were significant differences related to views of socialization; students taking in-person physics laboratories tended to value socialization more than students taking the course online. In another comparative study, Klein et al. investigated how physics students perceived the sudden shift to online learning during the pandemic.
They administered a questionnaire to 578 physics students from five universities in Germany, Austria, and Croatia, and they found that students who collected their own data using real equipment, as opposed to being given data or collecting data using simulations, felt that they gained more experimental skills [13]. Another study conducted during the pandemic by Borish et al. [20] found that students who were provided clear expectations, had enough time for their coursework, frequently worked in groups, and frequently had access to guidance from their instructors were more likely to report positive outcomes. It seems that many of the best practices studied during remote lab instruction due to the COVID-19 pandemic echo those reported prior to the pandemic. In a review of the pros and cons of online, remote, and distance science labs by Faulconer and Gruss in 2018 [24], they found that "active, visible, intentional engagement with students," "instructional design focused on developing students' skills in self-regulated learning" and a focus on "inquiry" are best practices for all laboratory courses and "nontraditional lab courses are no exception." These previous studies show that there is still much disagreement in the field when it comes to effective practices and benefits of remote lab courses-whether during a pandemic or not. While remote labs can provide increased flexibility and access for students, they may have negative learning outcomes in terms of skill development and socialization. In contrast to these studies, our goal is to highlight multiple approaches to remote lab instruction as described by the respondents to the instructor survey and describe challenges and successes from the instructors' perspective.

A. Survey design
The survey was divided into two main sections: first, we asked closed-response items where instructors indicated changes that occurred in the course from before to after the transition to remote instruction. These closed responses consisted of topics relating to lab structure and activities, course learning goals, student choices, equipment and technology resources, and scientific communication. Additional questions within these categories were added to capture the activities that may be unique to remote labs (e.g., using video conferencing tools). After each of the sections, instructors were given an open "other" option to describe any additional items that were not captured by the closed-response options, the inclusion of which was motivated by the fact that we had limited knowledge of what instructors were doing given the fast, emergency nature of the transition. An example of a set of questions probing student communication is shown in Fig. 1. The second half of the survey was comprised of a mix of closedand open-response questions asking about motivations, challenges, and successes of the remote class. For example, instructors were asked to "Describe the successful aspects of your remote lab class." All of the questions on the survey were optional.

B. Data collection
Survey volunteers were recruited through professional listservs related to laboratory instruction, as well as through an email to instructors currently administering the E-CLASS [23] in their courses. The emails included a link to the survey, which was administered via Qualtrics beginning on April 30, 2020, with the majority of responses from the instructors being received within the following two weeks. Because of the recruitment method, instructors who use E-CLASS represent 20% of the survey population. E-CLASS users are particularly interested in formative assessment of their course along the dimension of student epistemologies and attitudes around experimental physics and, therefore, may not be a representative sample of physics lab instructors as a whole.
The survey was completed for 129 courses by 106 unique instructors. A majority of the courses represented in the survey came from 4-year colleges (55%). Approximately 8% of the responses were about courses at 2-year colleges, 5% at master's granting institutions, and 32% at Ph.D. granting institutions. Most of the responses came from institutions in the United States (93%) with 60% of those being private not-for-profit institutions and 19% being minority serving institutions. From all the responses, 61% of courses were first-year (introductory) labs and 39% were beyond-first-year (BFY) labs. Approximately 30% of the labs were for primarily nonphysics or engineering majors, 60% were for primarily physics and engineering majors, and 10% to a mixture of majors. Most respondents switched to remote teaching part way through the term, though 17% of the courses were remote for the entire term (typically from quarter or trimester systems). It is important to note that none of the questions on the survey forced a response; yet, about half of the instructors who completed the survey gave lengthy open responses that ranged from a few sentences to multiple paragraphs detailing their experiences. We mention this because it is unusual for surveys like these to be filled out so completely; we posit that instructors were eager to share what they did and went through during the semester.

Quantitative methods
The first set of questions asked instructors to "Describe the activities in your lab course before and after transitioning to remote instruction," where they were then given a list of activities that might have been part of their course and two possible check boxes representing "Before remote instruction" and "After remote instruction" (an example question is shown in Fig. 1). We calculated the total number of courses that had a given activity for instruction before the transition and the total number of courses that used that activity after the transition to remote labs-the "before" and "after" responses were compared to identify significant changes based on the calculated uncertainty (overlap in the confidence interval). We calculated the uncertainty using the 95% binomial confidence interval for λ > 5 and the Poisson approximation for the binomial λ ≤ 5. In addition, we asked instructors to rank their agreement to statements about their motivations for the approach they chose, as well as the challenges they encountered. We used a 5-point Likert scale (from "strongly agree" to "strongly disagree") for these questions. We treated these data as interval data and assigned a number to each response as follows: "strongly disagree" ¼ 0, "disagree" ¼ 1, and "neutral" ¼ 2, "agree" ¼ 3, and "strongly agree" ¼ 4. From this scheme, we calculated means for the responses, with the uncertainty given as the standard error. Through this analysis, we are able to identify global trends of our survey population in order to determine common themes.

Qualitative methods
There were several open-response questions on the survey. To analyze responses to these questions, we developed two codebooks. First, we started with an a priori code book based on the categories of questions asked on the survey as a whole. Many of these main codes also have subcodes, which were created from the closed-response choices of the survey. Additional subcodes were added during the coding process as emergent codes. These emergent codes were created through a collaborative coding process. A. W. and K. O. independently coded a subset of the instructor open-response data (11 courses in total). The percent agreement between the two raters on these 11 responses was 97%. We report percent agreement instead of Cohen's kappa because the large number of subcodes, 99, along with the low prevalence of individual codes across the small dataset, can result in unreliable kappa values [25]. After establishing interrater reliability, the entirety of the data set was coded using the first code book. All additional emergent codes added after the initial interrater reliability were discussed and agreed upon by the research team.
As the successes identified by the instructors were critical to informing future instruction, we wanted to understand these in more detail. Therefore, we developed a second code book using the open-ended responses that had been coded as success in the first code book. This second code book was developed using only emergent coding and captured what instructors found to be successful, their metrics of success, qualifiers (e.g., at least some of the students enjoyed…), and things the instructors said they would continue using when they transitioned back to inperson instruction. The "what was successful" and "metrics of success" both had subcodes (16 and 23, respectively) that captured nuances of what the instructors considered successful and why. For example, an instructor wrote "Since the goal was primarily to explore physics concepts, I think the use of simulations helped us to still meet that goal." In this case, the "what was successful" were simulations and the "metric of success" was students learning physics concepts. J. H. and K. O. separately coded 20 responses that were coded as "success" using the first code book. These 20 responses were not used in the second code book creation process. The percent agreement between the two raters for the 20 responses was found to be 93%. Both codebooks are available in Appendix A.

Limitations
In developing the survey, we were aware that the closedresponse options we provided would unlikely be able to capture the full breadth of experiences faced by the instructors. However, through the analysis of the openresponse questions, as described in Sec. III C 2, we were able to supplement the closed-response data with instructor provided responses. The prevalence of these responses should be considered in the context that some were prompted and others unprompted, and so they may be considered a demonstration of existence.
The wording and interpretation of the survey questions was not validated beyond the research team due to the timesensitive nature of the research. Therefore, we cannot be certain that all instructors interpreted the questions in the way that we intended. For example, when responding to the question about learning goals instructors were given three options to choose: (i) primarily to reinforce physics concepts, (ii), primarily to develop lab skills, and (iii) both reinforce physics concepts and develop lab skills about equally. We interpret "lab skills" as broadly encompassing many goals such as modeling, designing experiments, developing technical and practical laboratory skills, analyzing and visualizing data, and communicating physics [26]. However, others may only consider "developing technical and practical laboratory skills," such as soldering or aligning optics as lab skills. Despite this limitation, the responses to the open-response questions reported in this work were consistent with our intentions when writing the survey. Furthermore, we focus primarily on these open responses in our analysis and thus instructors' interpretation of the questions, while still a limitation, do not impede our ability to draw meaningful conclusions from the results.
Another limitation of this study is that some instructors may not have had the time, energy, or ability to fill out an online survey due to increased stress and responsibilities due to the pandemic. Access to technology, having a quiet space to work, attending to family responsibilities, and dealing with both mental and physical healthcare were challenges not only for students, but for instructors as well. We did not collect the demographic information of the instructors surveyed, but we suspect that the sample of instructors might be biased in this way because women and marginalized people carried a disproportionate burden of stress and responsibilities during this pandemic time [27][28][29][30]. When drawing our conclusions in this study, we remain sensitive to these missing perspectives and hope that the results are interpreted with this in consideration.
In addition, we did not ask instructors to report on the race, ethnicity, gender, or socioeconomic status of their student populations because most instructors do not have easy access to this information or the permission to share it. The context and constraints faced by instructors vary based, in part, on the student population (e.g., instructors who teach students who are majority low income may have had a different set of considerations to take into account when determining how to structure a remote lab class); however, we did not want to further burden instructors with finding this information or needing to guess the demographic make-up of their students. This represents a limitation of closed-response portion of our study where we cannot compare the constraints faced by our instructors to other outside factors.

IV. RESULTS AND DISCUSSION
Through our mixed-method analysis of the instructor responses, we identified three common themes: (i) successful use of simulations, (ii) a shift of goals toward conceptual learning, (iii) and challenges fostering collaboration in the online environment. We identified these themes, answering RQ1, by triangulating instructor reported goals, challenges, and successes. Together, when discussing these three themes, we draw on past literature and research to postulate on how the transition to remote instruction could inform lab course design for both in-person and remote labs in the future to answer RQ2.

A. Successful use of simulations
As instruction moved online, labs-which often require data analysis and experimental design-turned to creative technological solutions. Instructors of 15 different courses used simulations successfully across many metrics of success, including remaining similar to the in-person experience (5 courses) and achieving learning goals of the course (4 courses). In addition, simulations were described as successful in a variety of different course types and for a mix of learning goals, see Tables I and II. Most of the instructors (11=15) were able to maintain their initial learning goals by using simulations. As shown in Appendices B 1 and B 4, after the transition, there was a large, significant increase in the use of simulation tools [31]. Although 15 courses is only a small percentage of those that used simulations-65 reported conducting lab activities through simulations (Appendix B 1) and 59 reported using a simulation tool (Appendix B 4), it is important to note, that unlike questions regarding equipment, technology, and lab activities which were closedresponse options on the survey, successes were asked as an open-ended question so does not speak to its insignificance. In fact, using simulations was one of the most common reported successes amongst the instructors who responded to the open-ended question about successes (Fig. 2).
An instructor from a medium-sized, introductory lab for nonphysics or engineering majors wrote, "The use of the photoelectric effect and blackbody PhET simulations was were able to still pick up the main physics of new circuit components." And that they viewed SPICE as an "extra skill" that students were able to learn. However, they noted that the students had already developed decent electronics lab skills (test and measurement skills, bread boarding, grounding, debugging, etc.) during the in-person half of the semester so the transition to using simulated circuits easier for students. Instructors reported using simulations as sources of data collection, making measurements, and learning physics content. Past research, conducted prior to the pandemic, found that simulations were useful for reinforcing physics concepts [32][33][34]. This is particularly true as some simulations have been developed to address specific and common student difficulties [35]. We saw in the survey responses that a larger percentage of courses emphasizing physics concepts as their primary learning goal after the transition to remote instruction used PhET simulations [36] (47.5%) than courses whose focus was learning lab skills during remote instruction (20.5%).
However, multiple instructors in the survey discussed the usefulness of other simulations for developing lab skills. Eleven instructors mentioned using simulations beyond PhET including: Fritzing [37], KET [38], MultisimLive [39], oPhysics [40], SPICE [41], MATLAB's Simulink [42], The Physics Aviary [43], and students coding their own simulations. Some simulations allow students to interact with models of physical phenomena via their computers or smartphones and engage in authentic decision making, data collection, and troubleshooting practices. These simulations tend to have larger parameter spaces for students to explore such as Pivot Interactives [44]-a hybrid of simulation and video analysis, where real experiments have been filmed with a variety of different parameter selections. These allow students to explore the real-world parameter space and, using overlaid measurement tools, perform measurements from the videos. An instructor from a small, BFY course for engineering and physics majors called simulations "valuable" and wrote, "I might use them as part of a class even with in-person learning." In addition, many electronics labs found circuit simulations such as SPICE, MATLAB's Simulink [42], or MultisimLive [39] particularly useful because students were able to build and model "real" circuits (with nonidealized performance). Since these tools are commonly used in industry, this also meant that students could still have an authentic lab experience and develop important lab skills. One instructor wrote, Given the original design of the lab activities, a combination of Fritzing [37] and MultisimLive [39] allowed students to practice many of the skills I had already planned to address.
While some instructors noted that simple simulations may not be able to replicate the complex aspects of performing experiments in real life, the example above of using Fritzing [37] may emulate more what working on circuit design is like for professionals, than compared to using simpler simulations.

B. Shift of learning goals
We found that, although the instructors described a range of motivations, most were driven by the desire to meet the course learning goals and to cover the same concepts as before remote instruction (Fig. 3). However, when asked about the broad learning goals of the class (developing skills, reinforcing concepts, or a mixture of both) for both before and after the transition, instructors reported switching more towards emphasizing physics concepts rather than lab skills. This was particularly true for instructors that, prior to the emergency remote instruction, had the course focus on both concepts and skills about equally (Fig. 4).
From the closed-response data, we cannot comment on the specific "skills" and "concepts" that various courses focused on. That is, it is difficult to know if the shift of courses that emphasized both concepts and skills prior to remote instruction to primarily learning physics concepts was because the skills learning goals were centered around using hands-on equipment (e.g., soldering), which students were unable to do remotely. However, based on the open responses, this was the case for some of the instructors. An instructor from a small, BFY course whose goal before and after the transition to remote instruction was to reinforce both skills and concepts equally wrote, "teaching lab skills involving hands-on use of equipment was not possible" and indicated that after the transition, they primarily reinforced concepts. Another instructor from a Ph.D. granting institution teaching a small, BFY course whose goal before and after the transition to remote instruction was to primarily reinforce skills wrote, One of the three course goals involves developing students' ability to use the tools and techniques that experimentalists use in the lab. This is pretty much impossible remotely.
However, as we see in Fig. 4, the majority of courses with primary learning goals associated with skills maintained those learning goals after the transition, with many people finding creative ways to focus on laboratory skills in the remote classes. Another survey respondent, an instructor from a master's granting institution teaching a BFY, small laboratory course for physics and engineering majors said, Even though no lab work occurred after remote instruction began, students had to rely on their notebooks and previous data collection to complete required oral presentations and written reports, both considered part of "lab skills" (i.e., experimental physics skills).
It is important to note that the above two quotes were both from BFY courses. Traditionally, BFY and first-year courses have very different learning goals with BFY courses more heavily emphasizing lab skill development [26,45].
More generally, these results align with past literature that finds many proponents of online labs value learning physics concepts (i.e., content and theory) where proponents of hands-on labs often value design skills and collaborative skills [4][5][6][7]. Perhaps the online environment may be easier to achieve the learning goals associated with learning physics concepts in contrast to lab skills.
FIG. 3. Instructors were asked to "Rank how much you agree with the following statements." We show the mean response from 121 survey responses and the uncertainty, which represents one standard error of the mean. We calculated the mean by assigning a response of strongly disagree ¼ 0, disagree ¼ 1, and neutral ¼ 2, agree ¼ 3, and strongly agree ¼ 4. Given the extenuating circumstances, pivoting the learning goals of a lab course to focus more on concepts may have been a reasonable, productive, and effective solution. However, as we look to lessons from this experience that we can take into future instruction of labs, developing lab skills is an important part of the undergraduate physics curriculum [26]. In fact, a past study found that physics laboratory courses that focus specifically on developing lab skills promote more expertlike beliefs about the nature of experimental physics than courses that focus either on reinforcing physics concepts or on both goals [46,47]. From the instructor responses, it seemed that the ability to maintain a focus on experimental skills during remote instruction depends on the resources available to students and instructors, as well as on what skills are considered important. The obvious challenge associated with remote lab instruction is the potential absence of hands-on interaction with measurement devices and experimental apparatus, particularly if the lab requires sophisticated and expensive equipment. Some classes were able to continue hands-on experimentation in Spring 2020 by sending equipment home to students or having students use resources from home, including the use of smartphone applications as measurement devices (Appendix A, Fig. 6). Additional instructors indicated that they were "looking into lab kits to be sent to students' homes" for future remote terms. However, many of these home lab kits do not allow for opportunity to learn how to work with lasers, detectors, vacuum pumps, lock-in amplifiers, cyrostats, and other complex experimental apparatus-an important aspect of many BFY physics labs [26].
Another common learning goal for labs includes developing skills associated with data and uncertainty analysis [48]. To do this, instructors sent student data that they had collected previously or that they generated for the purposes of the course. Alternatively, some instructors asked students to review data from scientific publications or publicly available data sets, since the development of data analysis skills does not necessarily require students to collect their own data. However, using previously collected or opensource data may diminish student understanding of how the data were acquired and how it should be interpreted. Instructors overcame this challenge by having students control equipment remotely, watch videos of the instructor(s) take data, or even have students provide the instructor with directions with how to collect the data. An instructor from a small, introductory lab did just this: Students actively instructing me in the conduct of and [sic] experiment that has video footage to analyze does not appear to be different for many labs goals than for them to do it themselves, at a surprisingly high level.
Still, the instructor felt that their students missed … actual manipulative skills that would come from handling the equipment, and some agency and executive function skills that are not exercised because the setting constrains choices to a smaller range than students would face when confronted with a document and equipment.
It is clear that some necessary physics lab skills are challenging to replicate completely in a remote environment. Nonetheless, some skills were less affected by the transition to remote instruction than others. For example, courses focusing on the development of scientific writing, reading, and presentation skills-especially writing lab reports, giving oral presentations, reading scientific papers, writing experimental proposals, and writing reviews of scientific papers-were able to continue doing this after the transition (see Appendix B 3).

C. Collaboration in an online environment
Collaboration is often an essential part of labs. The AAPT Recommendations for the Undergraduate Physics Laboratory Curriculum [26] suggest that one of the goals for students in physics labs should be to develop "interpersonal communication skills" through "teamwork and collaboration." In addition, as research and business is increasingly conducted in a global environment, many believe that it is essential for students to be prepared to engage in effective collaborations remotely within diverse groups [49][50][51]. However, in the survey, some instructors expressed that they were "concerned about the mechanisms of group work with the rapid transition to online" so they switched to mostly individual work. Because of logistical concerns, other instructors gave students the option to opt-out of group work. One instructor wrote, "After remote transition, students could work in groups or individually. I did not structure it either way. Before we transitioned, I structured it as group work." And the move to individual work was not isolated to only a few instructors; in a closed-response question, we asked the instructors whether students "worked in groups with other students" or "worked individually" before and after the transition to remote instruction. Figure 5 shows that before the transition to remote instruction, 82% of the courses reported that students only worked in groups. After the transition to remote instruction, only 24.8% of students exclusively worked in groups, while the percentage of courses where students worked only individually on labs increased to 55.6%.
From the open responses, the most common motivation for this change toward individual work was equity concerns. Because of the sudden nature of the transition, many instructors could not or did not want to require students to attend labs synchronously: We could not require that all the students perform the lab synchronously during the designated lab time.
(Some students had limitations on internet access). So we allowed students to complete the lab activity asynchronously within a 30 h time frame. This has led to some students opting to work alone with the data collection part and engage in group activity only at the report writing stage.
These decisions resonate with recent reports on the impact of the COVID-19 pandemic on college students that suggest that being a student emerged as a higher risk factor for loneliness during lockdown than usual [52] in addition to increased worry and grief [53]. Higher levels of social capital and sense of community are significantly associated with lower levels of loneliness [54]. Fostering group work in the online classroom can pose new challenges [55]; nonetheless, it can have overwhelming benefits including increased motivation, creativity, and reflection [56]-essentials during a time of increased isolation for students.
Five of the instructors found methods for successful student collaboration despite the challenging circumstances. One instructor said that the students continued to engage with the material and "even more deeply because of the considerable increase in messaging between the groups and myself due to the remote class." Small breakout rooms in Zoom seemed to help some with enabling collaboration. One instructor expected group work to be a larger issue in the remote setting, but found that it was not as challenging as expected "as long as I kept the groups to three students." Another said, Students could still work productively in groups trying to do sense-making activities-the zoom break out rooms (and my ability to pop-in and pop-out of those rooms to address the problems the students were grappling with worked better than I thought it would).
A fourth instructor wrote, "Group projects came out fine even given the challenges. Students all continued to participate at the same level, so no issues there." Finally, one instructor noted an increase in the students reaching out for help, "…student requests for assistance increased compared to traditional instruction. Students used GroupMe (online chat app used for course) to request Zoom meetings to go over topics." They noted that facilitating the course became a community effort and everyone in the course, including the students and instructors, had to work "together to ensure information was accessible to all and was updated in a timely fashion." The examples of successful aspects of remote lab courses seen in our dataset lead us to wonder how this experience may positively (or negatively) influence physics lab education beyond the pandemic. For example, online group work clearly posed a challenge for many instructors and one instructor wrote that they would have liked to have "resources for how to manage group work online." An increase in accessible resources describing some best practices in online group work could help instructors who are (a) interested in moving their lab courses online or (b) need to quickly switch to online labs in response to volatile weather, future pandemics, or other natural disasters.

D. Beyond Spring 2020 and conclusions
Together, we can use these results to reflect on how to best conduct physics labs in the future either in-person or remote (RQ2). Online education has grown steadily since the early 2000s due to new technologies, global adoption of the Internet, and a demand for a college-educated workforce [57][58][59][60]. Some are postulating that for college teaching and learning, there may be no return to normal since the COVID-19 pandemic will disrupt the notion that courses taught online are significantly worse than in-person learning [61]. It is reasonable to expect online learning in higher education will be in our future, and understanding best pedagogical practices in the online environment is essential-particularly for labs.
However, as shown in the instructor responses, there are many benefits to in-person labs that can be difficult or impossible to replicate in an online environment. Collaborating face to face with instructors and peers to troubleshoot technical equipment (Appendix B 2) is just one example of a staple of in-person physics labs that is essential to student growth and development as experimental physicists [26]. Perhaps these missed elements of in-person physics labs will spur a renewed interest from instructors and increase the emphasis on them during in-person labs. Nonetheless, it is important to realize that online education is likely to further increase in the future due to various institutional, economic, and societal pressures.
Although the rapid, unpredictable nature of this transition led to an extreme set of challenges, some of these issues with moving labs online will likely persist even without time constraints on the implementation of the courses. For example, how can we make remote labs a similar experience to in-person labs, which often rely on hands-on equipment? And without students in the classroom, how can we teach the necessary lab skills, and do so safely? Perhaps more importantly, should we even attempt to move labs to an online environment for the majority of students? Regardless, if we think most labs should return to being primarily in-person, could we make labs more accessible to students with disabilities or to those who do not have access to physical labs, such as at remote locations? Some of these questions have started to be answered through past work on science labs in general [4][5][6][7]9,22,24]. For example, in the 2018 review of online, remote, and distance science laboratory by Faulconer and Gruss, they found that online or remote labs and labs with home kits had benefits when it came to multiple access opportunities, extended access time, disability access, and safety. Additional studies, [8,12,14] which specifically looked at physics labs found that when it came to having a "similar experience," changes in students' epistemological beliefs were the same regardless of being in-person or remote. However, we believe more effort is needed to identify the best practices for remote physics labs, particularly BFY courses.
Through this work, we identified some tools used by instructors, such as focusing on scientific writing skills and using authentic simulation tools, to successfully implement lablike learning in a remote setting. However, similar to past literature [4][5][6][7][10][11][12][13][14][15][16][17], we found that approaches that worked in some institutional and classroom contexts were not equitable in others. To illustrate, an instructor from a small, beyond-first-year class teaching a modern physics lab said I could imagine a class where experiments are done by the students at home, but given the different life circumstances of students, the class would likely not be an equitable experience.
On the other hand, an instructor from another school teaching a medium-sized, introductory lab said, "I had them measure the focal length of their cell phone camera lens based on the recent paper… Worked well!" However successful, it is important to note that this solution could lead to inequitable experiences for students if they do not all own a cell phone with a camera.
While we have identified the common themes among the instructors experiences during the transition to remote instruction, there were other notable successes that individual instructors reported influencing their future labs. For example, some instructors used this as an opportunity to try a new curriculum that they hope to bring back to the in-person experience. Five instructors reported that they plan to incorporate simulations into the future in-person experience. In addition, one instructor implemented contract grading: So one day it hit me to try a grading contract, which has really minimized how much I have to formally grade. I give students feedback but they get credit for completion, so the grading burden is a lot smaller on me… I think when we go back to in person I am going to try some sort of hybrid so every single check in doesn't have to be graded, but students must show completion of everything to get some type of minimal grade.
Another instructor "took advantage of the free LabArchives" and said that "students gave positive feedback on that so I'm considering switching to e-notebooks next year." Lastly, we saw that one instructor was able to move from "cookbook" experiments that were "focused on making sure students saw a particular result" to a "guided research project" in the remote version. They found that student engagement actually increased in the remote format and the open-ended format allowed students to be "engaged in problem solving" and make "meaningful decisions" about the experimental processes.

V. CONCLUSIONS
This work presents the wide range of approaches that instructors employed in Spring 2020 in order to teach remote physics lab classes, and demonstrates some of the possible ways to successfully conduct remote labs, as well as some of the common challenges. We encourage future studies to continue analyzing the impact of remote labs on student learning, particularly from the student perspective. More work must be done to investigate the ability to achieve common physics lab learning goals, the impact on student development of professional collaborative skills, and student identity as experimental physicists in online environments.

Challenges
Includes fifteen subcodes such as personal life of the instructor or student, equity, and time to capture the types of challenges faced by the instructional team and students.

Successes
Coded whenever instructors discuss a success. These codes were later used to create second code book (see Appendix A, Table IV).
Resources that were helpful Coded when discussing helpful resources; no subcodes.
Resources that we would like Resources that would have been helpful but were not available; no subcodes.
Changes to lab Describes changes that instructors did make or would like to make in the following semester to the remote lab. This code includes seven subcodes which detail what the instructors would have like to change (e.g., course content or lab activities).

Collaboration
An emergent code for when instructors discussed students working in groups or individually.

Student engagement
An emergent code for when instructors discussed students' levels of engagement in the remote course.

Creativity
An emergent code for when instructors discussed students thinking creatively or when instructions described creativity as a learning goal. Metrics of success Includes 15 subcodes which describe why an instructor believed a certain element was successful. For example, one instructor found "synchronous sessions" to be successful because they had "good attendance and participation." In this case, synchronous labs would be coded as "What was successful" and student engagement would be coded as the "Metric of success."

Qualifiers
This was recorded to note when an instructor qualified a success. For example, "I hosted a review on Blackboard Collaborate that was successful, though not as good as in-person." This instructor found the learning management system to be successful, but qualified that it was still not as good as in-person labs.
Would use for in-person Coded when an instructor indicated that they would like to carry an activity or practice back into the inperson labs. For example, "The simulation labs were valuable. I might use them as part of a class even with in-person learning." code book. The second table (Table IV) provides descriptions of the success codes.

APPENDIX B: ADDITIONAL CLOSED-RESPONSE DATA
Here, we provide the data from the closed-response questions on the survey about what was done in the labs before and after the transition to remote instruction along with some descriptions highlighting important aspects of the data. In these figures, the y axis is the number of courses who reported using these activities after switching to remote instruction, except for the "other" category, which is the number of courses that did "other things" based on the open responses. The inset plot shows the breakdown of the other category. The green bars represent a statistically significant (when comparing the 95% binomial confidence intervals) increase in that activity after the transition to remote instruction. The red bars represent a statistically significant (when comparing the 95% binomial confidence intervals) decrease in that activity after the transition to remote instruction. The solid blue bars represent a change in that activity that was not statistically different from before the transition. We do not know how many instructors used activities in the other category before the transition, so they are denoted by striped blue bars. All significance and errors bars were calculated using the 95% binomial confidence interval.

Lab structure and activities
After the transition to remote instruction, many instructors changed both the lab structure and the activities in their courses. In particular, instructors did not have as many traditional guided labs (74 before the transition to 43 after the transition with a 95% binomial confidence interval of AE11) and they increased the number of asynchronous lab activities. These asynchronous activities included having students analyze data provided by the instructor and having students use simulations as replacements for the in-person lab activities (Fig. 6).
There were significant increases (green bars in Fig. 6) in activities analyzing instructor provided data, labs conducted through simulations, students watching videos of labs being conducted by instructional staff, students using household equipment to complete lab activities, and equipment being sent from the school to students' homes. In addition, instructors discussed in their open responses to the survey that they had students continue to work on projects from before the transition and focus on scientific communication (other category in Fig. 6). Given that many institutions transitioned to remote instruction in the middle FIG. 6. Instructor reported lab activities used after the transition to remote instruction in Spring 2020. of the Spring 2020 semester, some classes were able to pivot and extend projects that the students had already started. Others who could not conduct previously planned experiments due to the remote environment, opted to have students write review papers on a scientific topics.

Student choices and self-regulated learning experiences
Activities in an online learning environment are often very different from conventional lab classes. Many remote classes had fewer or no opportunities for students to interact face-to-face with their instructors and classmates. During asynchronous components of online classes, students are more responsible for their own learning, as they decide when, where, and for how long to work on course activities and assignments; therefore, self-regulated learning behaviors are especially important when taking online courses [62,63]. For remote physics labs, this could potentially result in added student decision making about their own analysis methods, lab procedures, troubleshooting experimental apparatus at home, building their own apparatus, and developing their own research questions. Figure 7 shows that after the transition to remote instruction, many instructors said that their students were able to continue to choose their own analysis method with no significant decrease compared to in-person learning.
Although the number of students choosing their own analysis methods remained the same from before to after the transition, we saw a significant drop in students troubleshooting problems with experimental setups-likely due to fewer hands-on activities. However, one instructor found that computer modeling of data allowed students to work on their troubleshooting skills without needing handson labs. Another instructor considered that in many fields of science, researchers do their science remotely (e.g., astronomers, high-energy physicists) when not physically constructing detectors. Although this instructor does not explicitly mention troubleshooting, it is true that many physicists who work with remote equipment engage in the troubleshooting processes-for example, the Hubble flawed mirror design [64].

Communication and collaboration
We asked instructors about engaging with various forms of scientific writing, reading, and presentation before and after the transition to remote instruction (Fig. 8). This scientific communication stayed relatively similar; however, there was a significant drop from 82 to 56 courses (with a 95% binomial confidence interval of AE11) in which students wrote in lab notebooks. In the open responses, instructors did not report on asking students to engage in any forms of scientific communication outside of what was captured in the closed responses. However, some instructors needed to adapt the technology used to the remote environment such as engaging in oral presentations, literature review, and using Google Docs.

Technology and equipment
As instruction moved online, most instructors who responded to our survey used video conferencing technology in order to interact with their students and hold their classes. However, beyond regular face-to-face (or, in this case, screen-to-screen) communication, labs often require data analysis and experimental design-aspects that can require creative technological solutions to move online.
After the transition, there was a significant increase in the reliance on PhET simulations, YouTube videos, and students building their own equipment at home (green bars in Fig. 9). There was a significant decrease in students using university-owned equipment (i.e., equipment or experimental setups made by the school for students). Additionally, we see in Fig. 9 that there were many other diverse technological solutions instructors used that were not captured by the closed-response portion of the survey. In the open responses, 20 instructors mentioned that they created their own videos for students.