Testing the Ability to Apply Mathematical Knowledge

Since the 1960s, the advocacy of teaching mathematics so as to be useful is not without hindrance in school curricula, partly due to the lack of appropriate assessment tools. Practical approaches have been accumulating quickly, but researchers showed that they are not satisfactory in testing students’ ability to apply mathematical knowledge, be they “word problems” in school textbooks, national tests, or large-scale international assessments. To understand the causes behind the dissatisfaction, there is a need to reveal (1) the theories that are used in the test designs, and (2) what the actual assessments are in various curricula. This motive leads to the purpose of the current study, which is to identify empirically consistent theories about students’ ability to apply; the results can be organized as a framework to analyze assessment tools such as PISA, as well as various curricular materials. Based on the current theories, a framework of assessment analysis is created in order to study the coverage of modeling steps of public assessment items. This study finds that, though many education systems have claims of introducing modeling and application into their curricula, high-stake assessments mostly involve a small fraction of the steps that are required in a full modeling cycle. It furthers an earlier result that certain textbooks, though claiming the importance of modeling, almost ignored the first and last steps of modeling. It is found in this study that public assessments are even more limited: most test items that are supposed to test students’ knowledge of application involve only one or two steps of modeling. Furthermore, the tool “modeling spectrum” that is used in the analysis does not only reveal how modeling steps are covered, but can also assists educators to improve or create problems with modeling and application.