116343.pdf (370.75 kB)
Leveraging out-of-the-box retrieval models to improve mental health support
Version 2 2023-08-07, 09:06
Version 1 2023-06-10, 06:14
conference contribution
posted on 2023-08-07, 09:06 authored by Julie WeedsJulie Weeds, Theo Rummer-DowningThis work compares the performance of several information retrieval (IR) models in the search for relevant mental health documents based on relevance to forum post queries from a fully-moderated online mental health service. Three different architectures are assessed: a sparse lexical model, BM25, is used as a baseline, alongside two neural SBERT-based architectures - the bi-encoder and the cross-encoder. We highlight the credibility of using pretrained language models (PLMs) out-of-the-box, without an additional fine-tuning stage, to achieve high retrieval quality across a limited set of resources. Error analysis of the ranking results suggested PLMs make errors on documents which contain so called red-herrings - words which are semantically related but irrelevant to the query - whereas human judgements were found to suffer when queries are vague and present no clear information need. Further, we show that bias towards an author’s writing style within a PLM affects retrieval quality and, therefore, can impact on the success of mental health support if left unaddressed.
History
Publication status
- Published
File Version
- Published version
Journal
16th International Joint Conference on Biomedical Engineering Systems and TechnologiesISSN
2184-4305Publisher
SCITEPRESS Digital LibraryPublisher URL
External DOI
Page range
64-73Event name
16th International Joint Conference on Biomedical Engineering Systems and TechnologiesEvent location
Lisbon, PortugalEvent type
conferenceEvent date
16th-18th February 2023ISBN
9789897586316Department affiliated with
- Informatics Publications
Full text available
- Yes
Peer reviewed?
- Yes
Legacy Posted Date
2023-02-15First Compliant Deposit (FCD) Date
2023-02-14Usage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC