BarrettDienesSeth_PSYCHMETH2013.pdf (379.19 kB)
Measures of metacognition on signal-detection theoretic models
journal contributionposted on 2023-06-08, 16:46 authored by Adam BarrettAdam Barrett, Zoltan DienesZoltan Dienes, Anil SethAnil Seth
Analysing metacognition, specifically knowledge of accuracy of internal perceptual, memorial or other knowledge states, is vital for many strands of psychology, including determining the accuracy of feelings of knowing, and discriminating conscious from unconscious cognition. Quantifying metacognitive sensitivity is however more challenging than quantifying basic stimulus sensitivity. Under popular signal detection theory (SDT) models for stimulus classification tasks, approaches based on type II receiver-operator characteristic (ROC) curves or type II d-prime risk confounding metacognition with response biases in either the type I (classification) or type II (metacognitive) tasks. A new approach introduces meta-d': the type I d-prime that would have led to the observed type II data had the subject used all the type I information. Here we (i) further establish the inconsistency of the type II d-prime and ROC approaches with new explicit analyses of the standard SDT model, and (ii) analyse, for the first time, the behaviour of meta-d' under non-trivial scenarios, such as when metacognitive judgments utilize enhanced or degraded versions of the type I evidence. Analytically, meta-d' values typically reflect the underlying model well, and are stable under changes in decision criteria; however, in relatively extreme cases meta-d' can become unstable. We explore bias and variance of in-sample measurements of meta-d' and supply MATLAB code for estimation in general cases. Our results support meta-d' as a useful measure of metacognition, and provide rigorous methodology for its application. Our recommendations are useful for any researchers interested in assessing metacognitive accuracy.
- Accepted version
PublisherAmerican Psychological Association
Department affiliated with
- Informatics Publications
NotesThis article may not exactly replicate the final version published in the APA journal. It is not the copy of record.
Full text available