A Cognitively Diagnostic Modeling Approach to Diagnosing Misconceptions and Subskills

The objective of the present project was to propose a new methodology for measuring misconceptions and subskills simultaneously using diagnostic information available from incorrect alternatives in multiple-choice tests designed for that purpose. Misconceptions are systematic and persistent errors that represent a learned intentional incorrect response (Brown & VanLehn, 1980; Ozkan & Ozkan, 2012). In prior research, Lee and Corter (2011) found that classification accuracy for their Bayesian Network misconception diagnosis models improved when latent higher-order subskills and specific wrong answers were included. Here, these contributions are adapted to a cognitively diagnostic measurement approach using the multiple-choice Deterministic Inputs Noisy “And” Gate (MC-DINA) model, first developed by de la Torre (2009b), by specifying dependencies between attributes to measure latent misconceptions and subskills simultaneously. A simulation study was conducted employing the proposed methodology (referred to as MC-DINA-H) across sample sizes (500, 1000, 2,000, and 5,000 examinees) and test lengths (15, 30, and 60 items) conditions. Eight attributes (4 misconceptions and 4 subskills) were included in the main simulation study. Attribute classification accuracy of the MC-DINA-H was compared to four less complex models and was found to more accurately classify attributes only when the attributes were relatively frequently required by multiple-choice options in the diagnostic assessment. The findings suggest that each attribute should be required by at least 15-20 percent of options in the diagnostic assessment.