Diagnostic classification
models

A brief introduction

W. Jake Thompson, Ph.D.

Conceptual foundations

  • Traditional assessments and psychometric models measure an overall skill or ability
  • Assume a continuous latent trait

A normal distribution with team logos from the National Women's Soccer League overlayed.

Traditional methods

  • The output is a weak ordering of teams due to error in estimates
    • Confident Portland Thorns are the best
    • Not confident who is second best (OL Reign, North Carolina Courage, San Diego Wave)
  • Limited in the types of questions that can be answered.
    • Why is Kansas City so low?
    • What aspects are teams competent/proficient in?
    • How much skill is “enough” to pass?

Soccer example

  • Rather than measuring overall team strength, we can break soccer down into set of skills or attributes
    • Finishing
    • Possession
    • Defending
    • Goalkeeping

  • Attributes are categorical, often dichotomous (e.g., proficient vs. non-proficient)

Diagnostic classification models

  • DCMs place individuals into groups according to proficiency of multiple attributes
finishing possession defending goalkeeping
Xmark
Check
Xmark
Xmark
Check
Check
Check
Xmark
Check
Xmark
Xmark
Check
Check
Check
Check
Xmark

Answering more questions

  • Why is Kansas City so low?
    • Poor finishing, defending, and goalkeeping
  • What aspects are teams competent/proficient in?
    • DCMs provide classifications directly

Diagnostic psychometrics

  • Designed to be multidimensional
  • No continuum of student achievement
  • Categorical constructs
    • Usually binary (e.g., master/nonmaster, proficient/not proficient)
  • Several different names in the literature
    • Diagnostic classification models (DCMs)
    • Cognitive diagnostic models (CDMs)
    • Skills assessment models
    • Latent response models
    • Restricted latent class models

When are DCMs appropriate?

Success depends on:

  1. Domain definitions
    • What are the attributes we’re trying to measure?
    • Are the attributes measurable (e.g., with assessment items)?
  2. Alignment of purpose between assessment and model
    • Is classification the purpose?

Example applications

  • Educational measurement: The competencies that student is or is not proficient in
    • Latent knowledge, skills, or understandings
    • Used for tailored instruction and remediation
  • Psychiatric assessment: The DSM criteria that an individual meets
    • Broader diagnosis of a disorder

When are DCMs not appropriate?

  • When the goal is to place individuals on a scale

  • DCMs do not distinguish within classes