University of Sussex

File(s) not publicly available

SemEval-2007 Task 10: English Lexical Substitution Task

posted on 2023-06-07, 20:32 authored by Diana McCarthy, Roberto Navigli
In this paper we describe the English Lexical Substitution task for SemEval. In the task, annotators and systems find an alternative substitute word or phrase for a target word in context. The task involves both finding the synonyms and disambiguating the context. Participating systems are free to use any lexical resource. There is a subtask which requires identifying cases where the word is functioning as part of a multiword in the sentence and detecting what that multiword is.


Publication status

  • Published


The Association for Computational Linguistics



Presentation Type

  • paper

Event name

Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007)

Event location

Prague, Czech Republic

Event type


Department affiliated with

  • Informatics Publications


Originality: This paper describes a new approach for evaluating word sense disambiguation algorithms and the lexical resources that these algorithms rely on by evaluating the ability of a system to find a good substitute for a word in context. In contrast to previous work, the method does not require a man-made inventory. Rigour: We collected annotations from 5 annotators for over 2000 sentences and evaluated systems using our new methodology in an international competition (SemEval 2007). Significance: This paper makes a major contribution because the task makes no requirement on the inventory of word meaning and the task reflects capabilities that will benefit NLP applications such as question answering. Impact: The competition was run in 2007 and we had 8 different international teams participating, with a total of 10 systems. This is a high number of systems for a brand new task. We anticipate that this research will spawn new directions in the evaluation of lexical sense inventories and lexical disambiguation. Outlet: SemEval is the fourth of a successful series of international evaluation workshops, initially called SENSEVAL. This workshop was held at ACL 2007. Full papers were reviewed.

Full text available

  • No

Peer reviewed?

  • Yes

Legacy Posted Date


Usage metrics

    University of Sussex (Publications)


    No categories selected


    Ref. manager