Anatomy of a physics test: Validation of the physics items on the Texas Assessment of Knowledge and Skills

We report the results of an analysis of the Texas Assessment of Knowledge and Skills (TAKS) designed to determine whether the TAKS is a valid indicator of whether students know and can do physics at the level necessary for success in future coursework, STEM careers, and life in a technological society. We categorized science items from the 2003 and 2004 10th and 11th grade TAKS by content area(s) covered, knowledge and skills required to select the correct answer, and overall quality. We also analyzed a 5000 student sample of item-level results from the 2004 11th grade exam, performing full-information factor analysis, calculating classical test indices, and determining each item's response curve using item response theory. Triangulation of our results revealed strengths and weaknesses of the different methods of analysis. The TAKS was found to be only weakly indicative of physics preparation and we make recommendations for increasing the validity of standardized physics testing.

Figure 1

Item response curve for item 45 of the 2004 11th grade TAKS.

Figure 2

Item response curve for item 41 of the 2004 11th grade TAKS.

Figure 3

(Color) TPAT expert ratings plotted against the IRT “$a$” (discrimination) parameter for the physics items from the 2004 exit-level TAKS.