Assessing the interactivity and prescriptiveness of faculty professional development workshops: The real-time professional development observation tool

Professional development workshops are one of the primary mechanisms used to help faculty improve their teaching, and draw in many STEM instructors every year. Although workshops serve a critical role in changing instructional practices within our community, we rarely assess workshops through careful consideration of how they engage faculty. Initial evidence suggests that workshop leaders often overlook central tenets of education research that are well established in classroom contexts, such as the role of interactivity in enabling student learning [S. Freeman et al., Proc. Natl. Acad. Sci. U.S.A. 111, 8410 (2014)]. As such, there is a need to develop more robust, evidence-based models of how best to support faculty learning in professional development contexts, and to actively support workshop leaders in relating their design decisions to familiar ideas from other educational contexts. In response to these needs, we have developed an observation tool, the real-time professional development observation tool (R-PDOT), to document the form and focus of faculty engagement during workshops. In this paper, we describe the motivation and methodological considerations behind the development of the R-PDOT, justify our decisions to highlight particular aspects of workshop sessions, and demonstrate how the R-PDOT can be used to analyze three sessions from the Physics and Astronomy New Faculty Workshop. We also justify the accessibility and potential utility of the R-PDOT output as a reflective tool using preliminary data from interviews with workshop leaders, and consider the roles the R-PDOT could play in supporting future research on faculty professional development.

Figure 1

Color key for the type-of-engagement (left) and focus-of-engagement (right) R-PDOT codes.

Figure 2

Summative plots showing how much time was spent enacting each R-PDOT code during three approximately 1-h long sessions of the Physics and Astronomy New Faculty Workshop, labeled $A$, $B$, and $C$. The pie charts show the type of faculty’s engagement as defined in Table 1, where authoritative communicative approaches are shown in brown and dialogic approaches are shown in teal; the bar charts shows the focus of faculty’s engagement as defined in Table 3. Further discussion of the color scheme can be found in Sec. 5b.

Figure 3

Time line representations of the R-PDOT data for session $B$, showing how the type (top) and focus (bottom) of faculty’s engagement shifted during the session. The colors are consistent with those in Figs. 1 and 2, where again, brown indicates authoritative communicative approaches and teal indicates dialogic communicative approaches for the type of engagement, and the focus-of-engagement colors indicate loose clusters.

Figure 4

Time line representations of the R-PDOT data for session $C$.