Examining evolving performance on the Force Concept Inventory using factor analysis

The application of factor analysis to the Force Concept Inventory (FCI) has proven to be problematic. Some studies have suggested that factor analysis of test results serves as a helpful tool in assessing the recognition of Newtonian concepts by students. Other work has produced at best ambiguous results. For the FCI administered as a pre- and post-test, we see factor analysis as a tool by which the changes in conceptual associations made by our students may be gauged given the evolution of their response patterns. This analysis allows us to identify and track conceptual linkages, affording us insight as to how our students have matured due to instruction. We report on our analysis of 427 pre- and post-tests. The factor models for the pre- and post-tests are explored and compared along with the methodology by which these models were fit to the data. The post-test factor pattern is more aligned with an expert’s interpretation of the questions’ content, as it allows for a more readily identifiable relationship between factors and physical concepts. We discuss this evolution in the context of approaching the characteristics of an expert with force concepts. Also, we find that certain test items do not significantly contribute to the pre- or post-test factor models and attempt explanations as to why this is so. This may suggest that such questions may not be effective in probing the conceptual understanding of our students.

Figure 1

The scree plots shown for the (a) pretest and (b) post-test cases.

Figure 2

The (partial) scree plots resulting from the parallel analysis for the pretest (a) and post-test (b) cases. As shown, for each case, five eigenvalues from the original data’s correlation matrix were found to be larger than the eigenvalues of the correlation matrix for the simulated data. This suggests that there are five factors for each case, pre- and post-test. For the post-test case, we actually decided to use six factors (please see text for explanation). The other eigenvalues for the original data take on values lower than those for the simulated data. Each eigenvalue for the simulated data is calculated at a 95% confidence level.