University of Sussex
Witchel_smiling_trigger-substrate-model_text_2018-06-12.pdf (486.98 kB)
Download file

A trigger-substrate model for smiling during an automated formative quiz: engagement is the substrate, not frustration

Download (486.98 kB)
conference contribution
posted on 2023-06-09, 14:53 authored by Harry WitchelHarry Witchel, Harry L Claxton, Daisy C Holmes, Thomas T Ranji, Joe D Chalkley, Carlos P Santos, Carina Westling, Michel F Valstar, Matt Celuszak, Patrick Fagan
INTRODUCTION: Automated tutoring systems aim to respond to the learner’s cognitive state in order to maintain engagement. The end-user’s state might be inferred by interactive timings, bodily movements or facial expressions. Problematic computerized stimuli are known to cause smiling during periods of frustration. METHODS: Forty-four seated, healthy participants (age range 18-35, 18 male) used a handheld trackball to answer a computer-presented, formative, 3-way multiple choice geography quiz, with 9 questions, lasting a total of 175 seconds. Frontal facial videos (10 Hz) were collected with a webcam and processed for facial expressions by CrowdEmotion using a pattern recognition algorithm. Interactivity was recorded by a keystroke logger (Inputlog 5.2). Subjective responses were collected immediately after each quiz using a panel of visual analogue scales (VAS). RESULTS: Smiling was fie-fold enriched during the instantaneous feedback segments of the quiz, and this was correlated with VAS ratings for engagement but not with happiness or frustration. Nevertheless, smiling rate was significantly higher after wrong answers compared to correct ones, and frustration was correlated with the number of questions answered incorrectly. CONCLUSION: The apparent disconnect between the increased smiling during incorrect answers but the lack of correlation between VAS frustration and smiles suggests a trigger-substrate model where engagement is the permissive substrate, while the noises made by the quiz after wrong answers may be the trigger.


Publication status

  • Published

File Version

  • Published version


Proceedings of the 36th European Conference on Cognitive Ergonomics


Association of Computing Machinery

Page range


Article number


Event name

ECCE 2018 - European Conference on Cognitive Ergonomics

Event location

Utrecht University, The Netherlands

Event type


Event date

5-7 September, 2018



Department affiliated with

  • BSMS Neuroscience Publications

Full text available

  • Yes

Peer reviewed?

  • Yes

Legacy Posted Date


First Open Access (FOA) Date


First Compliant Deposit (FCD) Date


Usage metrics

    University of Sussex (Publications)


    No categories selected