University of Sussex

File(s) not publicly available

A competency-based mixed-methods approach to evaluating a leadership development intervention in the NHS

posted on 2023-06-09, 19:30 authored by Emma RussellEmma Russell, J Yarker, L Waugh, Marc FullmanMarc Fullman, A Memon
Evaluating the success of interventions in applied organisational settings poses a number of problems for work and organisational psychologists. It is increasingly recognised that random control trials are practically difficult to implement in real-world intervention designs. Our aim is to demonstrate that intervention evaluation research can still be rigorous and informative, even when applied organisational interventions are limited in their design. In this study, an evaluation of a 9-month leadership development programme, with 41 senior NHS commissioners, was conducted. However, with non-random allocation of participants, and no control comparison group available, we developed a competency-based, mixed-methods approach to evaluate the programme’s success. This served to overcome issues traditionally associated with within-person intervention evaluations, whereby changes occurring from the pre to the post-intervention period cannot confidently be attributed to the intervention itself (Shadish, Cook & Campbell, 2002). Our study offers a thorough and novel approach by: (1) applying a repeated-measures design to capture information on intervention content and style of delivery at each of the 7 sessions that formed the leadership development programme; (2) operating a mixed-method design, incorporating structured observations, self-report competency assessments and evaluations, other-report competency assessments, and objective data about line-manager support; and, (3) taking a longitudinal approach (months, rather than days or weeks) to measuring intervention impact, rarely applied when using momentary-based repeated measures designs. Data will be analysed using multi-level random coefficient modelling, (by ML-Win), and will be available by EAWOP 2017. Implications for evaluation research, specifically from a socio-technical systems perspective are discussed.


Publication status

  • Published

Event name

18th Congress of the European Association of Work and Organizational Psychology (EAWOP): Enabling Change through Work and Organizational Psychology

Event location


Event type


Event date

17-20 May 2017

Department affiliated with

  • Business and Management Publications

Full text available

  • No

Peer reviewed?

  • No

Legacy Posted Date


Usage metrics

    University of Sussex (Publications)


    No categories selected


    Ref. manager